[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:16 dist-test-slave-9gcw krb5kdc[31313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:16 dist-test-slave-9gcw krb5kdc[31313](info): set up 2 sockets
Aug 08 10:34:16 dist-test-slave-9gcw krb5kdc[31313](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 10:34:17.863291 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.010s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:17 dist-test-slave-9gcw krb5kdc[31313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:34:18.005796 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:39855
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:39855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:18.134110 31329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:18.134491 31329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:18.134599 31329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:18.139232 31329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:18.139321 31329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:18.139364 31329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:18.139424 31329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:18.139465 31329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:18.145741 31329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:39855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:39855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:18.147468 31329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:18.149309 31329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:18.161041 31339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:18.161113 31329 server_base.cc:1034] running on GCE node
W20260808 10:34:18.160972 31336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:18.160974 31337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:18.162066 31329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:18.163828 31329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:18.165040 31329 hybrid_clock.cc:648] HybridClock initialized: now 1786185258165024 us; error 61 us; skew 500 ppm
Aug 08 10:34:18 dist-test-slave-9gcw krb5kdc[31313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:18.168334 31329 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:34:18.169461 31329 webserver.cc:460] Webserver started at http://127.30.146.190:42581/ using document root <none> and password file <none>
I20260808 10:34:18.170071 31329 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:18.170168 31329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:18.170544 31329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:18.172766 31329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "ce38b57b380c451e841a48bd67ea6a45"
format_stamp: "Formatted at 2026-08-08 10:34:18 on dist-test-slave-9gcw"
server_key: "4856e46ddd8e858c3aaaa62c633edd57"
server_key_iv: "2d9353385f390e91d72156d9b20d92f9"
server_key_version: "encryptionkey@0"
I20260808 10:34:18.173427 31329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "ce38b57b380c451e841a48bd67ea6a45"
format_stamp: "Formatted at 2026-08-08 10:34:18 on dist-test-slave-9gcw"
server_key: "4856e46ddd8e858c3aaaa62c633edd57"
server_key_iv: "2d9353385f390e91d72156d9b20d92f9"
server_key_version: "encryptionkey@0"
I20260808 10:34:18.177700 31329 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:34:18.180418 31346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:18.181897 31329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:18.182044 31329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "ce38b57b380c451e841a48bd67ea6a45"
format_stamp: "Formatted at 2026-08-08 10:34:18 on dist-test-slave-9gcw"
server_key: "4856e46ddd8e858c3aaaa62c633edd57"
server_key_iv: "2d9353385f390e91d72156d9b20d92f9"
server_key_version: "encryptionkey@0"
I20260808 10:34:18.182166 31329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:18.207692 31329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:18.211395 31329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:18.211694 31329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:18.225328 31329 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:39855
I20260808 10:34:18.225358 31401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:39855 every 8 connection(s)
I20260808 10:34:18.226637 31329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:18.230674 31402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:18.234767 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 31329
I20260808 10:34:18.234963 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:18.236958 31306 external_mini_cluster.cc:1428] Setting key 627cce47f7a4afa610808c064914f77d
I20260808 10:34:18.238520 31402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45: Bootstrap starting.
I20260808 10:34:18.241712 31402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:18.242740 31402 log.cc:824] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:18.245479 31402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45: No bootstrap required, opened a new log
I20260808 10:34:18.251151 31402 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce38b57b380c451e841a48bd67ea6a45" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39855 } }
I20260808 10:34:18.251446 31402 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:18.251508 31402 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce38b57b380c451e841a48bd67ea6a45, State: Initialized, Role: FOLLOWER
I20260808 10:34:18.252063 31402 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce38b57b380c451e841a48bd67ea6a45" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39855 } }
I20260808 10:34:18.252235 31402 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:18.252323 31402 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:18.252439 31402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:18.253496 31402 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce38b57b380c451e841a48bd67ea6a45" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39855 } }
I20260808 10:34:18.253835 31402 leader_election.cc:302] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce38b57b380c451e841a48bd67ea6a45; no voters:
I20260808 10:34:18.254179 31402 leader_election.cc:288] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:18.254258 31407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:18.254498 31407 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [term 1 LEADER]: Becoming Leader. State: Replica: ce38b57b380c451e841a48bd67ea6a45, State: Running, Role: LEADER
I20260808 10:34:18.254895 31407 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce38b57b380c451e841a48bd67ea6a45" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39855 } }
I20260808 10:34:18.255609 31402 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:18.257472 31409 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce38b57b380c451e841a48bd67ea6a45. Latest consensus state: current_term: 1 leader_uuid: "ce38b57b380c451e841a48bd67ea6a45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce38b57b380c451e841a48bd67ea6a45" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39855 } } }
I20260808 10:34:18.257519 31408 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce38b57b380c451e841a48bd67ea6a45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce38b57b380c451e841a48bd67ea6a45" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39855 } } }
I20260808 10:34:18.257622 31408 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:18.257622 31409 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:18.258178 31419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 10:34:18 dist-test-slave-9gcw krb5kdc[31313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:18.261927 31419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:18.266952 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:18.241425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48812 (local address 127.30.146.190:39855)
0808 10:34:18.242133 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:18.242152 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:18.243209 (+ 1057us) server_negotiation.cc:407] Connection header received
0808 10:34:18.248257 (+ 5048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:18.248293 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:18.248613 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:18.248883 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:18.252401 (+ 3518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:18.253601 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:18.255135 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:18.255668 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:18.255827 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:18.261266 (+ 5439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:18.261273 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:18.261297 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:18.261325 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:18.263288 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:18.263837 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:18.263843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:18.263848 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:18.263962 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:18.264321 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:18.264325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:18.264327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:18.264842 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:18.265104 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:18.265711 (+ 607us) server_negotiation.cc:299] Negotiation successful
0808 10:34:18.265989 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":219,"threads_started":1}
I20260808 10:34:18.269093 31419 catalog_manager.cc:1347] Generated new cluster ID: 28fc9fa8cd964c3d8dda3da06c38d2e3
I20260808 10:34:18.269176 31419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:18.281288 31419 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:18.282672 31419 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:18.293471 31419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce38b57b380c451e841a48bd67ea6a45: Generated new TSK 0
I20260808 10:34:18.294476 31419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:34:18.332186 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39855
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:18.499294 31432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:18.499595 31432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:18.499684 31432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:18.503847 31432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:18.503914 31432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:18.503975 31432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:18.509617 31432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:18.510811 31432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:18.511833 31432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:18.523595 31440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:18.523653 31437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:18.523689 31432 server_base.cc:1034] running on GCE node
W20260808 10:34:18.523613 31438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:18.524281 31432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:18.524938 31432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:18.526108 31432 hybrid_clock.cc:648] HybridClock initialized: now 1786185258526094 us; error 69 us; skew 500 ppm
Aug 08 10:34:18 dist-test-slave-9gcw krb5kdc[31313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:18.529850 31432 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:34:18.530853 31432 webserver.cc:460] Webserver started at http://127.30.146.129:33723/ using document root <none> and password file <none>
I20260808 10:34:18.531451 31432 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:18.531548 31432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:18.531802 31432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:18.534217 31432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "a1ac68b11a274cc786a3675c606626bc"
format_stamp: "Formatted at 2026-08-08 10:34:18 on dist-test-slave-9gcw"
server_key: "0fb802a53d89a276580ccb0393e8cf66"
server_key_iv: "40f68ab84ee0a70339730aefb68d1586"
server_key_version: "encryptionkey@0"
I20260808 10:34:18.535171 31432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "a1ac68b11a274cc786a3675c606626bc"
format_stamp: "Formatted at 2026-08-08 10:34:18 on dist-test-slave-9gcw"
server_key: "0fb802a53d89a276580ccb0393e8cf66"
server_key_iv: "40f68ab84ee0a70339730aefb68d1586"
server_key_version: "encryptionkey@0"
I20260808 10:34:18.540099 31432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 10:34:18.543107 31447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:18.544539 31432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:18.544782 31432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "a1ac68b11a274cc786a3675c606626bc"
format_stamp: "Formatted at 2026-08-08 10:34:18 on dist-test-slave-9gcw"
server_key: "0fb802a53d89a276580ccb0393e8cf66"
server_key_iv: "40f68ab84ee0a70339730aefb68d1586"
server_key_version: "encryptionkey@0"
I20260808 10:34:18.544934 31432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:18.577515 31432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:18.581261 31432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:18.581554 31432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:18.582468 31432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:18.583587 31432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:18.583665 31432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:18.583746 31432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:18.583796 31432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:18.603377 31432 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:37415
I20260808 10:34:18.603474 31569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:37415 every 8 connection(s)
I20260808 10:34:18.604483 31432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:18.611863 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 31432
I20260808 10:34:18.612016 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:18.612334 31306 external_mini_cluster.cc:1428] Setting key 2592288f17a3885c7226e129b9c2e54c
Aug 08 10:34:18 dist-test-slave-9gcw krb5kdc[31313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:18.622030 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:18.606887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:44519 (local address 127.30.146.190:39855)
0808 10:34:18.607069 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:18.607073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:18.608992 (+ 1919us) server_negotiation.cc:407] Connection header received
0808 10:34:18.610448 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:18.610452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:18.610519 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:18.610631 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:18.612060 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:18.612913 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:18.614485 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:18.614760 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:18.614894 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:18.618092 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:18.618095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:18.618098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:18.618124 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:18.619789 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:18.620357 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:18.620365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:18.620366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:18.620443 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:18.620824 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:18.620827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:18.620828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:18.621012 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:18.621123 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:18.621702 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 10:34:18.621847 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 10:34:18.622927 31574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:18.607388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39855 (local address 127.30.146.129:44519)
0808 10:34:18.608647 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:18.608693 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:18.610084 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:18.610830 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:18.610837 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:18.611317 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:18.611847 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:18.611862 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:18.613073 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:18.613077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:18.614339 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:18.614351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:18.614944 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:18.614949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:18.615223 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:18.615963 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:18.615980 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:18.617736 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:18.619923 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:18.619929 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:18.619945 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:18.620218 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:18.620539 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:18.620543 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:18.620545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:18.620692 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:18.621169 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:18.621177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:18.621445 (+ 268us) client_negotiation.cc:769] Sending connection context
0808 10:34:18.621637 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 10:34:18.621988 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"thread_start_us":774,"threads_started":1}
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:34:18.624507 31572 heartbeater.cc:347] Connected to a master server at 127.30.146.190:39855
I20260808 10:34:18.624778 31572 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:18.625385 31572 heartbeater.cc:511] Master 127.30.146.190:39855 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:34:18.627633 31364 ts_manager.cc:194] Registered new tserver with Master: a1ac68b11a274cc786a3675c606626bc (127.30.146.129:37415)
I20260808 10:34:18.629278 31364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:44519
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:34:18.645547 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39855
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:18.800005 31578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:18.800310 31578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:18.800402 31578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:18.804733 31578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:18.804807 31578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:18.804911 31578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:18.810631 31578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:18.811791 31578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:18.812929 31578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:18.823627 31584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:18.823662 31586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:18.823671 31583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:18.823982 31578 server_base.cc:1034] running on GCE node
I20260808 10:34:18.824491 31578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:18.825248 31578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:18.826422 31578 hybrid_clock.cc:648] HybridClock initialized: now 1786185258826406 us; error 29 us; skew 500 ppm
Aug 08 10:34:18 dist-test-slave-9gcw krb5kdc[31313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:18.829324 31578 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:34:18.830149 31578 webserver.cc:460] Webserver started at http://127.30.146.130:39023/ using document root <none> and password file <none>
I20260808 10:34:18.830690 31578 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:18.830747 31578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:18.830915 31578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:18.832938 31578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "16c7a91fbf1947cba0f18d971d8188e6"
format_stamp: "Formatted at 2026-08-08 10:34:18 on dist-test-slave-9gcw"
server_key: "b8a52e41dc37324074c9cf80dc211b5a"
server_key_iv: "26bb0f6a707f9d54296dbfeafec45ff0"
server_key_version: "encryptionkey@0"
I20260808 10:34:18.833609 31578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "16c7a91fbf1947cba0f18d971d8188e6"
format_stamp: "Formatted at 2026-08-08 10:34:18 on dist-test-slave-9gcw"
server_key: "b8a52e41dc37324074c9cf80dc211b5a"
server_key_iv: "26bb0f6a707f9d54296dbfeafec45ff0"
server_key_version: "encryptionkey@0"
I20260808 10:34:18.837795 31578 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:34:18.840440 31593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:18.841794 31578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:18.841918 31578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "16c7a91fbf1947cba0f18d971d8188e6"
format_stamp: "Formatted at 2026-08-08 10:34:18 on dist-test-slave-9gcw"
server_key: "b8a52e41dc37324074c9cf80dc211b5a"
server_key_iv: "26bb0f6a707f9d54296dbfeafec45ff0"
server_key_version: "encryptionkey@0"
I20260808 10:34:18.842036 31578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:18.862917 31578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:18.866127 31578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:18.866348 31578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:18.867019 31578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:18.867897 31578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:18.867942 31578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:18.868014 31578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:18.868057 31578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:18.884621 31578 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:44421
I20260808 10:34:18.884748 31716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:44421 every 8 connection(s)
I20260808 10:34:18.885938 31578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:18.894440 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 31578
I20260808 10:34:18.894536 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:18.894886 31306 external_mini_cluster.cc:1428] Setting key 928f046bf61d186a5ee3e5aaf60b3170
Aug 08 10:34:18 dist-test-slave-9gcw krb5kdc[31313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:18.904049 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:18.888777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:49291 (local address 127.30.146.190:39855)
0808 10:34:18.889000 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:18.889008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:18.890636 (+ 1628us) server_negotiation.cc:407] Connection header received
0808 10:34:18.891813 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:18.891817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:18.891876 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:18.891992 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:18.893602 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:18.894132 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:18.895634 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:18.895935 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:18.896039 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:18.899857 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:18.899864 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:18.899871 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:18.899906 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:18.901839 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:18.902407 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:18.902414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:18.902419 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:18.902534 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:18.902896 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:18.902900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:18.902902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:18.903108 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:18.903222 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:18.903747 (+ 525us) server_negotiation.cc:299] Negotiation successful
0808 10:34:18.903879 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:34:18.905027 31719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:18.889307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39855 (local address 127.30.146.130:49291)
0808 10:34:18.890462 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:18.890509 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:18.891586 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:18.892169 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:18.892176 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:18.892748 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:18.893443 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:18.893460 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:18.894294 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:18.894302 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:18.895498 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:18.895508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:18.896112 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:18.896116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:18.896397 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:18.897238 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:18.897267 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:18.899593 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:18.901981 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:18.901987 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:18.902000 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:18.902270 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:18.902636 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:18.902638 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:18.902640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:18.902775 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:18.903220 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:18.903229 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:18.903494 (+ 265us) client_negotiation.cc:769] Sending connection context
0808 10:34:18.903692 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 10:34:18.904068 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"thread_start_us":623,"threads_started":1}
I20260808 10:34:18.906517 31717 heartbeater.cc:347] Connected to a master server at 127.30.146.190:39855
I20260808 10:34:18.906806 31717 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:18.907358 31717 heartbeater.cc:511] Master 127.30.146.190:39855 requested a full tablet report, sending...
I20260808 10:34:18.908584 31364 ts_manager.cc:194] Registered new tserver with Master: 16c7a91fbf1947cba0f18d971d8188e6 (127.30.146.130:44421)
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:34:18.909919 31364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:49291
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:34:18.932178 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39855
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:19.092141 31722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:19.092442 31722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:19.092494 31722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:19.097254 31722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:19.097332 31722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:19.097404 31722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:19.103863 31722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:19.105106 31722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:19.106317 31722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:19.117844 31728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:19.117821 31730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:19.117959 31722 server_base.cc:1034] running on GCE node
W20260808 10:34:19.117862 31727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:19.118551 31722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:19.119185 31722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:19.120357 31722 hybrid_clock.cc:648] HybridClock initialized: now 1786185259120340 us; error 29 us; skew 500 ppm
Aug 08 10:34:19 dist-test-slave-9gcw krb5kdc[31313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185259, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:19.123355 31722 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:34:19.124291 31722 webserver.cc:460] Webserver started at http://127.30.146.131:35451/ using document root <none> and password file <none>
I20260808 10:34:19.124890 31722 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:19.124986 31722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:19.125290 31722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:19.127511 31722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "afcd4ed135a347e795f539d0cc92df0e"
format_stamp: "Formatted at 2026-08-08 10:34:19 on dist-test-slave-9gcw"
server_key: "87c666c4d107d4bd2b7c684418403420"
server_key_iv: "70b1d093e937c5447e3a853011213b54"
server_key_version: "encryptionkey@0"
I20260808 10:34:19.128118 31722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "afcd4ed135a347e795f539d0cc92df0e"
format_stamp: "Formatted at 2026-08-08 10:34:19 on dist-test-slave-9gcw"
server_key: "87c666c4d107d4bd2b7c684418403420"
server_key_iv: "70b1d093e937c5447e3a853011213b54"
server_key_version: "encryptionkey@0"
I20260808 10:34:19.132194 31722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:34:19.134902 31737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:19.136334 31722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:19.136488 31722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "afcd4ed135a347e795f539d0cc92df0e"
format_stamp: "Formatted at 2026-08-08 10:34:19 on dist-test-slave-9gcw"
server_key: "87c666c4d107d4bd2b7c684418403420"
server_key_iv: "70b1d093e937c5447e3a853011213b54"
server_key_version: "encryptionkey@0"
I20260808 10:34:19.136620 31722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:19.151837 31722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:19.155253 31722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:19.155555 31722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:19.156386 31722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:19.157507 31722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:19.157579 31722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:19.157653 31722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:19.157706 31722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:19.175438 31722 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:45475
I20260808 10:34:19.175458 31862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:45475 every 8 connection(s)
I20260808 10:34:19.176481 31722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:19.181450 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 31722
I20260808 10:34:19.181584 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:19.181850 31306 external_mini_cluster.cc:1428] Setting key adec4ceefb2dfe970156426e326a1e0a
Aug 08 10:34:19 dist-test-slave-9gcw krb5kdc[31313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185259, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:19.192252 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:19.180918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:55881 (local address 127.30.146.190:39855)
0808 10:34:19.181054 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:19.181058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:19.181082 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:34:19.181899 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:19.181903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:19.181966 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:19.182048 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:19.183553 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:19.184097 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:19.185508 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:19.185742 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:19.185858 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:19.188727 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:19.188730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:19.188732 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:19.188758 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:19.190224 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:19.190786 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:19.190789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:19.190790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:19.190872 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:19.191188 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:19.191191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:19.191192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:19.191372 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:19.191450 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:19.192004 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 10:34:19.192115 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:34:19.193104 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:19.179442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39855 (local address 127.30.146.131:55881)
0808 10:34:19.180456 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:19.180502 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:19.181664 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:19.182230 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:19.182236 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:19.182781 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:19.183396 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:19.183410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:19.184242 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:19.184245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:19.185371 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:19.185381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:19.185919 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:19.185923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:19.186190 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:19.186870 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:19.186887 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:19.188548 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:19.190334 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:19.190339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:19.190351 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:19.190690 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:19.190953 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:19.190955 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:19.190956 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:19.191087 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:19.191466 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:19.191471 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:19.191791 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 10:34:19.191949 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 10:34:19.192225 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"thread_start_us":609,"threads_started":1}
I20260808 10:34:19.194365 31863 heartbeater.cc:347] Connected to a master server at 127.30.146.190:39855
I20260808 10:34:19.194693 31863 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:19.195235 31863 heartbeater.cc:511] Master 127.30.146.190:39855 requested a full tablet report, sending...
I20260808 10:34:19.196496 31363 ts_manager.cc:194] Registered new tserver with Master: afcd4ed135a347e795f539d0cc92df0e (127.30.146.131:45475)
I20260808 10:34:19.197091 31363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:55881
I20260808 10:34:19.208324 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:34:19 dist-test-slave-9gcw krb5kdc[31313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185259, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:34:19 dist-test-slave-9gcw krb5kdc[31313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185259, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:34:19.238919 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:19.227929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.30.146.129:37415)
0808 10:34:19.228280 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:19.228288 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:19.228345 (+ 57us) server_negotiation.cc:407] Connection header received
0808 10:34:19.228497 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:19.228503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:19.228721 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:19.228874 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:19.229703 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:19.230961 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:19.231794 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:19.232342 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:19.232449 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:19.235107 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:19.235118 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:19.235135 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:19.235164 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:19.237086 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:19.237559 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:19.237565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:19.237567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:19.237664 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:19.237916 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:19.237919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:19.237921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:19.238218 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:19.238408 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:19.238569 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 10:34:19.238735 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":131,"threads_started":1}
W20260808 10:34:19.239941 31477 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40972
Aug 08 10:34:19 dist-test-slave-9gcw krb5kdc[31313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:34:19 dist-test-slave-9gcw krb5kdc[31313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:34:19.267755 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:19.259285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.30.146.129:37415)
0808 10:34:19.259475 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:19.259481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:19.259618 (+ 137us) server_negotiation.cc:407] Connection header received
0808 10:34:19.259816 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:19.259819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:19.259864 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:19.259952 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:19.260671 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:19.261262 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:19.262101 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:19.262343 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:19.262454 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:19.265026 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:19.265029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:19.265031 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:19.265055 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:19.266360 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:19.266743 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:19.266745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:19.266746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:19.266840 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:19.267068 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:19.267070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:19.267071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:19.267230 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:19.267332 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:19.267527 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 10:34:19.267628 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:34:19.268946 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 31432
I20260808 10:34:19.277048 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 31578
I20260808 10:34:19.287410 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 31722
I20260808 10:34:19.294757 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 31329
2026-08-08T10:34:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2598 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:19 dist-test-slave-9gcw krb5kdc[31878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:19 dist-test-slave-9gcw krb5kdc[31878](info): set up 2 sockets
Aug 08 10:34:19 dist-test-slave-9gcw krb5kdc[31878](info): commencing operation
krb5kdc: starting...
W20260808 10:34:20.409343 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:20 dist-test-slave-9gcw krb5kdc[31878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:34:20.545131 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:32971
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:32971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:20.663012 31891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:20.663354 31891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:20.663429 31891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:20.667469 31891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:20.667568 31891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:20.667603 31891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:20.667642 31891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:20.667673 31891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:20.672963 31891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:32971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:32971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:20.674122 31891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:20.675240 31891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:20.684122 31900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:20.684258 31891 server_base.cc:1034] running on GCE node
W20260808 10:34:20.684195 31897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:20.684122 31898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:20.685084 31891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:20.686417 31891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:20.687587 31891 hybrid_clock.cc:648] HybridClock initialized: now 1786185260687562 us; error 40 us; skew 500 ppm
Aug 08 10:34:20 dist-test-slave-9gcw krb5kdc[31878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185260, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:20.690693 31891 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:34:20.691597 31891 webserver.cc:460] Webserver started at http://127.30.146.190:45175/ using document root <none> and password file <none>
I20260808 10:34:20.692168 31891 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:20.692229 31891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:20.692399 31891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:20.694607 31891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "e70aeb024f114b77902c316eaa5e0e1b"
format_stamp: "Formatted at 2026-08-08 10:34:20 on dist-test-slave-9gcw"
server_key: "50ebb30b5142b070f497b2acf64370d0"
server_key_iv: "ae9601adc252dd3e9ee781e5c3c65fc4"
server_key_version: "encryptionkey@0"
I20260808 10:34:20.695142 31891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "e70aeb024f114b77902c316eaa5e0e1b"
format_stamp: "Formatted at 2026-08-08 10:34:20 on dist-test-slave-9gcw"
server_key: "50ebb30b5142b070f497b2acf64370d0"
server_key_iv: "ae9601adc252dd3e9ee781e5c3c65fc4"
server_key_version: "encryptionkey@0"
I20260808 10:34:20.699381 31891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 10:34:20.701951 31907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:20.703286 31891 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:34:20.703433 31891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "e70aeb024f114b77902c316eaa5e0e1b"
format_stamp: "Formatted at 2026-08-08 10:34:20 on dist-test-slave-9gcw"
server_key: "50ebb30b5142b070f497b2acf64370d0"
server_key_iv: "ae9601adc252dd3e9ee781e5c3c65fc4"
server_key_version: "encryptionkey@0"
I20260808 10:34:20.703552 31891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:20.734750 31891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:20.738481 31891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:20.738715 31891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:20.746567 31891 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:32971
I20260808 10:34:20.746963 31965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:32971 every 8 connection(s)
I20260808 10:34:20.747639 31891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:20.750615 31966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:20.753192 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 31891
I20260808 10:34:20.753301 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:20.753588 31306 external_mini_cluster.cc:1428] Setting key 7ac199217b689a5adebd9886dc695afa
I20260808 10:34:20.757134 31966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b: Bootstrap starting.
I20260808 10:34:20.759954 31966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:20.760823 31966 log.cc:824] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:20.762782 31966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b: No bootstrap required, opened a new log
Aug 08 10:34:20 dist-test-slave-9gcw krb5kdc[31878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:20.766273 31966 raft_consensus.cc:348] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70aeb024f114b77902c316eaa5e0e1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32971 } }
I20260808 10:34:20.766546 31966 raft_consensus.cc:374] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:20.766630 31966 raft_consensus.cc:729] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70aeb024f114b77902c316eaa5e0e1b, State: Initialized, Role: FOLLOWER
I20260808 10:34:20.767175 31966 consensus_queue.cc:260] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70aeb024f114b77902c316eaa5e0e1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32971 } }
I20260808 10:34:20.767369 31966 raft_consensus.cc:388] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:20.767462 31966 raft_consensus.cc:482] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:20.767563 31966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:20.768796 31966 raft_consensus.cc:504] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70aeb024f114b77902c316eaa5e0e1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32971 } }
I20260808 10:34:20.769188 31966 leader_election.cc:302] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e70aeb024f114b77902c316eaa5e0e1b; no voters:
I20260808 10:34:20.769553 31966 leader_election.cc:288] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:20.769682 31972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:20.769939 31972 raft_consensus.cc:686] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [term 1 LEADER]: Becoming Leader. State: Replica: e70aeb024f114b77902c316eaa5e0e1b, State: Running, Role: LEADER
I20260808 10:34:20.770172 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:20.755173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.30.146.190:32971)
0808 10:34:20.755657 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:20.755684 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:20.755730 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:34:20.756578 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:20.756601 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:20.757022 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:20.757409 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:20.758110 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:20.759595 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:20.760513 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:20.761246 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:20.761401 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:20.763948 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:20.763961 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:20.763975 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:20.764001 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:20.766626 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:20.767172 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:20.767177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:20.767181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:20.767294 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:20.767734 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:20.767738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:20.767739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:20.768275 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:20.768477 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:20.768713 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 10:34:20.769046 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":137,"threads_started":1}
I20260808 10:34:20.770248 31972 consensus_queue.cc:237] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70aeb024f114b77902c316eaa5e0e1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32971 } }
I20260808 10:34:20.770941 31966 sys_catalog.cc:564] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:20.773058 31973 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e70aeb024f114b77902c316eaa5e0e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70aeb024f114b77902c316eaa5e0e1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32971 } } }
I20260808 10:34:20.773535 31973 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:20.773300 31974 sys_catalog.cc:455] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e70aeb024f114b77902c316eaa5e0e1b. Latest consensus state: current_term: 1 leader_uuid: "e70aeb024f114b77902c316eaa5e0e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e70aeb024f114b77902c316eaa5e0e1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32971 } } }
I20260808 10:34:20.773900 31974 sys_catalog.cc:458] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b [sys.catalog]: This master's current role is: LEADER
W20260808 10:34:20.775924 31986 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:34:20.776008 31986 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:34:20.776079 31987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:20.776791 31987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:20.782500 31987 catalog_manager.cc:1347] Generated new cluster ID: 9ae88710184f4577971c4d67a680ff82
I20260808 10:34:20.782570 31987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:20.809329 31987 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:20.810940 31987 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:20.830116 31987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e70aeb024f114b77902c316eaa5e0e1b: Generated new TSK 0
I20260808 10:34:20.831525 31987 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:34:20.885787 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32971
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:21.052126 31993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:21.052580 31993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:21.052644 31993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:21.057603 31993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:21.057682 31993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:21.057751 31993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:21.064141 31993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32971
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:21.065785 31993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:21.067241 31993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:21.078807 31998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:21.078861 31993 server_base.cc:1034] running on GCE node
W20260808 10:34:21.078832 32001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:21.078809 31999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:21.079617 31993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:21.080370 31993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:21.081533 31993 hybrid_clock.cc:648] HybridClock initialized: now 1786185261081504 us; error 100 us; skew 500 ppm
Aug 08 10:34:21 dist-test-slave-9gcw krb5kdc[31878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185261, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:21.085212 31993 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:34:21.086294 31993 webserver.cc:460] Webserver started at http://127.30.146.129:44253/ using document root <none> and password file <none>
I20260808 10:34:21.087074 31993 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:21.087204 31993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:21.087551 31993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:21.090279 31993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "e0fc54654439470cb4dbc394b81d54ba"
format_stamp: "Formatted at 2026-08-08 10:34:21 on dist-test-slave-9gcw"
server_key: "9902954a9926530ada62d74f670d3931"
server_key_iv: "edeae7fcbdd45b2738c2a622da67352a"
server_key_version: "encryptionkey@0"
I20260808 10:34:21.091077 31993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "e0fc54654439470cb4dbc394b81d54ba"
format_stamp: "Formatted at 2026-08-08 10:34:21 on dist-test-slave-9gcw"
server_key: "9902954a9926530ada62d74f670d3931"
server_key_iv: "edeae7fcbdd45b2738c2a622da67352a"
server_key_version: "encryptionkey@0"
I20260808 10:34:21.095492 31993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:34:21.098292 32009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:21.099540 31993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:21.099689 31993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "e0fc54654439470cb4dbc394b81d54ba"
format_stamp: "Formatted at 2026-08-08 10:34:21 on dist-test-slave-9gcw"
server_key: "9902954a9926530ada62d74f670d3931"
server_key_iv: "edeae7fcbdd45b2738c2a622da67352a"
server_key_version: "encryptionkey@0"
I20260808 10:34:21.099817 31993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:21.127105 31993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:21.131479 31993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:21.131803 31993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:21.132586 31993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:21.133701 31993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:21.133807 31993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:21.133950 31993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:21.134020 31993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:21.146332 31993 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:42183
I20260808 10:34:21.146387 32132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:42183 every 8 connection(s)
I20260808 10:34:21.147578 31993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:21.156543 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 31993
I20260808 10:34:21.156668 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:21.157073 31306 external_mini_cluster.cc:1428] Setting key b328bf60b30c7920f048fd654d27131b
Aug 08 10:34:21 dist-test-slave-9gcw krb5kdc[31878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185261, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:21.166697 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.149879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:54079 (local address 127.30.146.190:32971)
0808 10:34:21.150090 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:21.150096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:21.151330 (+ 1234us) server_negotiation.cc:407] Connection header received
0808 10:34:21.153957 (+ 2627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:21.153961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:21.154045 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:21.154147 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:21.155881 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.156561 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.158471 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.158719 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.158850 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.162101 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:21.162113 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:21.162115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:21.162149 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:21.164042 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:21.164718 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:21.164726 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:21.164727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:21.164806 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:21.165197 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:21.165205 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:21.165206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:21.165434 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:21.165604 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:21.166231 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 10:34:21.166433 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 10:34:21.167680 32135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.150508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:32971 (local address 127.30.146.129:54079)
0808 10:34:21.151103 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:21.151161 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:21.153667 (+ 2506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:21.154369 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:21.154379 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:21.155035 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:21.155672 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.155698 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.156778 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.156782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.158209 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.158227 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.158894 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.158898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.159077 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.159883 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:21.159911 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:21.161902 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:21.164189 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:21.164197 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:21.164223 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:21.164575 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:21.164901 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:21.164905 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:21.164908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:21.165065 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:21.165625 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:21.165644 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:21.165979 (+ 335us) client_negotiation.cc:769] Sending connection context
0808 10:34:21.166170 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 10:34:21.166637 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":163,"threads_started":1}
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:34:21.169428 32133 heartbeater.cc:347] Connected to a master server at 127.30.146.190:32971
I20260808 10:34:21.169775 32133 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:21.170428 32133 heartbeater.cc:511] Master 127.30.146.190:32971 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:34:21.172873 31928 ts_manager.cc:194] Registered new tserver with Master: e0fc54654439470cb4dbc394b81d54ba (127.30.146.129:42183)
I20260808 10:34:21.174368 31928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:54079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:34:21.193567 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32971
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:21.335412 32138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:21.335784 32138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:21.335834 32138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:21.341624 32138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:21.341749 32138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:21.341851 32138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:21.349503 32138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32971
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:21.351073 32138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:21.352736 32138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:21.367884 32143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:21.367924 32146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:21.367899 32144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:21.368479 32138 server_base.cc:1034] running on GCE node
I20260808 10:34:21.369444 32138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:21.370411 32138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:21.371644 32138 hybrid_clock.cc:648] HybridClock initialized: now 1786185261371609 us; error 46 us; skew 500 ppm
Aug 08 10:34:21 dist-test-slave-9gcw krb5kdc[31878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185261, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:21.375685 32138 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:34:21.376981 32138 webserver.cc:460] Webserver started at http://127.30.146.130:46877/ using document root <none> and password file <none>
I20260808 10:34:21.377890 32138 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:21.378036 32138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:21.378458 32138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:21.381451 32138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1"
format_stamp: "Formatted at 2026-08-08 10:34:21 on dist-test-slave-9gcw"
server_key: "ad44a1e18cffcfab4e8ca52993dd6f0d"
server_key_iv: "65bc47f583b61d502d1094393a45c885"
server_key_version: "encryptionkey@0"
I20260808 10:34:21.382108 32138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1"
format_stamp: "Formatted at 2026-08-08 10:34:21 on dist-test-slave-9gcw"
server_key: "ad44a1e18cffcfab4e8ca52993dd6f0d"
server_key_iv: "65bc47f583b61d502d1094393a45c885"
server_key_version: "encryptionkey@0"
I20260808 10:34:21.387233 32138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 10:34:21.390286 32155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:21.391913 32138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:34:21.392165 32138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1"
format_stamp: "Formatted at 2026-08-08 10:34:21 on dist-test-slave-9gcw"
server_key: "ad44a1e18cffcfab4e8ca52993dd6f0d"
server_key_iv: "65bc47f583b61d502d1094393a45c885"
server_key_version: "encryptionkey@0"
I20260808 10:34:21.392288 32138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:21.421854 32138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:21.426903 32138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:21.427150 32138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:21.427829 32138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:21.429366 32138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:21.429440 32138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:21.429517 32138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:21.429558 32138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:21.442229 32138 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:45321
I20260808 10:34:21.442252 32274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:45321 every 8 connection(s)
I20260808 10:34:21.443498 32138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:21.453557 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 32138
I20260808 10:34:21.453697 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:21.453959 31306 external_mini_cluster.cc:1428] Setting key 876e8bcba6d5e58164a68f03b9f74527
Aug 08 10:34:21 dist-test-slave-9gcw krb5kdc[31878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185261, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:21.462522 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.446092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:59703 (local address 127.30.146.190:32971)
0808 10:34:21.446278 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:21.446283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:21.447416 (+ 1133us) server_negotiation.cc:407] Connection header received
0808 10:34:21.449125 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:21.449129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:21.449272 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:21.449394 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:21.451022 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.451827 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.453404 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.453632 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.453786 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.458099 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:21.458103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:21.458106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:21.458133 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:21.459940 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:21.460698 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:21.460702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:21.460704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:21.460792 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:21.461215 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:21.461221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:21.461223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:21.461435 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:21.461554 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:21.462123 (+ 569us) server_negotiation.cc:299] Negotiation successful
0808 10:34:21.462264 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 10:34:21.463687 32277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.446571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:32971 (local address 127.30.146.130:59703)
0808 10:34:21.447180 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:21.447253 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:21.448862 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:21.449572 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:21.449579 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:21.450105 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:21.450773 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.450804 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.452009 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.452012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.453256 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.453271 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.453839 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.453843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.454033 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.455343 (+ 1310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:21.455361 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:21.457818 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:21.460148 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:21.460167 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:21.460194 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:21.460538 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:21.460894 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:21.460897 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:21.460899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:21.461072 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:21.461555 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:21.461563 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:21.461861 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 10:34:21.462061 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 10:34:21.462543 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":147,"threads_started":1}
I20260808 10:34:21.465418 32275 heartbeater.cc:347] Connected to a master server at 127.30.146.190:32971
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:34:21.465705 32275 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:21.466327 32275 heartbeater.cc:511] Master 127.30.146.190:32971 requested a full tablet report, sending...
I20260808 10:34:21.467748 31928 ts_manager.cc:194] Registered new tserver with Master: 8fe81a83fc0a4cb1be178b71fa01d3a1 (127.30.146.130:45321)
I20260808 10:34:21.468655 31928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:59703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:34:21.490780 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32971
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:21.640050 32281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:21.640436 32281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:21.640486 32281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:21.646064 32281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:21.646183 32281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:21.646329 32281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:21.657939 32281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32971
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:21.659796 32281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:21.661464 32281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:21.677295 32286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:21.677261 32287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:21.677291 32291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:21.677798 32281 server_base.cc:1034] running on GCE node
I20260808 10:34:21.678563 32281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:21.679488 32281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:21.680771 32281 hybrid_clock.cc:648] HybridClock initialized: now 1786185261680743 us; error 54 us; skew 500 ppm
Aug 08 10:34:21 dist-test-slave-9gcw krb5kdc[31878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185261, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:21.685607 32281 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:34:21.687249 32281 webserver.cc:460] Webserver started at http://127.30.146.131:41163/ using document root <none> and password file <none>
I20260808 10:34:21.688125 32281 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:21.688290 32281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:21.688573 32281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:21.691951 32281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73"
format_stamp: "Formatted at 2026-08-08 10:34:21 on dist-test-slave-9gcw"
server_key: "65a9434154228da256f11584c1b8b2e7"
server_key_iv: "d51d5bd49d929ebdcbc7da9ebee93cd8"
server_key_version: "encryptionkey@0"
I20260808 10:34:21.692983 32281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73"
format_stamp: "Formatted at 2026-08-08 10:34:21 on dist-test-slave-9gcw"
server_key: "65a9434154228da256f11584c1b8b2e7"
server_key_iv: "d51d5bd49d929ebdcbc7da9ebee93cd8"
server_key_version: "encryptionkey@0"
I20260808 10:34:21.699640 32281 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 10:34:21.703881 32298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:21.706049 32281 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:34:21.706218 32281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73"
format_stamp: "Formatted at 2026-08-08 10:34:21 on dist-test-slave-9gcw"
server_key: "65a9434154228da256f11584c1b8b2e7"
server_key_iv: "d51d5bd49d929ebdcbc7da9ebee93cd8"
server_key_version: "encryptionkey@0"
I20260808 10:34:21.706360 32281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:21.749137 32281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:21.754545 32281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:21.754935 32281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:21.755847 32281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:21.757076 32281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:21.757177 32281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 10:34:21.757268 32281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:21.757316 32281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:21.781733 32281 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:44159
I20260808 10:34:21.782047 32420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:44159 every 8 connection(s)
I20260808 10:34:21.783142 32281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:21.792755 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 32281
I20260808 10:34:21.792972 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:21.793447 31306 external_mini_cluster.cc:1428] Setting key 4f83696b7e08a7887cdb3faeeb9298cd
Aug 08 10:34:21 dist-test-slave-9gcw krb5kdc[31878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185261, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:21.808531 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.785846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:48983 (local address 127.30.146.190:32971)
0808 10:34:21.786040 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:21.786044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:21.787534 (+ 1490us) server_negotiation.cc:407] Connection header received
0808 10:34:21.790040 (+ 2506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:21.790044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:21.790132 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:21.790316 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:21.792174 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.793325 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.795345 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.795684 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.795851 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.803402 (+ 7551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:21.803405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:21.803407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:21.803437 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:21.805505 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:21.806137 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:21.806143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:21.806145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:21.806276 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:21.806751 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:21.806758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:21.806760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:21.807158 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:21.807308 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:21.808144 (+ 836us) server_negotiation.cc:299] Negotiation successful
0808 10:34:21.808275 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:34:21.809873 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.786404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:32971 (local address 127.30.146.131:48983)
0808 10:34:21.787324 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:21.787382 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:21.789774 (+ 2392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:21.790542 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:21.790550 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:21.791016 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:21.791920 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.791953 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.793503 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.793514 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.795163 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.795181 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.795910 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.795915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.796371 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.800856 (+ 4485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:21.800894 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:21.803178 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:21.805657 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:21.805664 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:21.805683 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:21.806001 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:21.806377 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:21.806391 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:21.806393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:21.806604 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:21.807309 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:21.807319 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:21.807744 (+ 425us) client_negotiation.cc:769] Sending connection context
0808 10:34:21.808091 (+ 347us) client_negotiation.cc:240] Negotiation successful
0808 10:34:21.808555 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":287,"threads_started":1}
I20260808 10:34:21.811326 32421 heartbeater.cc:347] Connected to a master server at 127.30.146.190:32971
I20260808 10:34:21.811687 32421 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:21.812537 32421 heartbeater.cc:511] Master 127.30.146.190:32971 requested a full tablet report, sending...
I20260808 10:34:21.814244 31928 ts_manager.cc:194] Registered new tserver with Master: 1dd0f5b9717f4e90a5167ec5ad0c5d73 (127.30.146.131:44159)
I20260808 10:34:21.815080 31928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:48983
I20260808 10:34:21.821621 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:34:21.837625 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.827428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.30.146.190:32971)
0808 10:34:21.827638 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:21.827646 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:21.828146 (+ 500us) server_negotiation.cc:407] Connection header received
0808 10:34:21.828353 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:21.828355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:21.828399 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:21.828505 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:21.829736 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.830414 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.831494 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.831754 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.831840 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.833700 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:21.833703 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:21.833705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:21.833730 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:21.835614 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:21.836088 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:21.836091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:21.836092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:21.836212 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:21.836513 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:21.836515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:21.836516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:21.836809 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:21.836957 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:21.837255 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 10:34:21.837390 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:34:21.845687 31928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:34:21.850147 31928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:21.875006 32434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.865085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:42183 (local address 127.0.0.1:33532)
0808 10:34:21.865792 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:21.865849 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:21.865983 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:21.867421 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:21.867428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:21.867461 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:21.867789 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.867799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.870241 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.870247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.872295 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.872317 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.874226 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.874231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.874451 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.874474 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:21.874562 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:34:21.874736 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":96,"thread_start_us":352,"threads_started":1}
I20260808 10:34:21.875062 32438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.865918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:44159 (local address 127.0.0.1:43188)
0808 10:34:21.866645 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:21.866662 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:21.867042 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:21.867729 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:21.867743 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:21.867761 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:21.868566 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.868574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.870676 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.870681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.872574 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.872601 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.874564 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.874569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.874699 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.874720 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:34:21.874849 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 10:34:21.874947 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":241,"thread_start_us":157,"threads_started":1}
I20260808 10:34:21.875054 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.866515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.30.146.130:45321)
0808 10:34:21.867266 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:21.867303 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:21.867326 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:34:21.867406 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:21.867411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:21.867638 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:21.867782 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:21.868795 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.870088 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.872577 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.873902 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.874095 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.874395 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:21.874622 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 10:34:21.874734 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":202,"threads_started":1}
I20260808 10:34:21.875562 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.866112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:45321 (local address 127.0.0.1:47474)
0808 10:34:21.866706 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:21.866727 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:21.866968 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:21.868126 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:21.868130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:21.868163 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:21.868580 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.868593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.870392 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.870396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.872324 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:21.872337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.874066 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.874081 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:21.874348 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.874396 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 10:34:21.874589 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 10:34:21.875414 (+ 825us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":923,"thread_start_us":207,"threads_started":1}
I20260808 10:34:21.875559 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.866057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33532 (local address 127.30.146.129:42183)
0808 10:34:21.866661 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:21.866668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:21.866820 (+ 152us) server_negotiation.cc:407] Connection header received
0808 10:34:21.866957 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:21.866968 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:21.867247 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:21.867436 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:21.868026 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.870060 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.872523 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.873945 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.874751 (+ 806us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.874963 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:21.875020 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 10:34:21.875389 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":212,"threads_started":1}
I20260808 10:34:21.875947 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:21.865507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.30.146.131:44159)
0808 10:34:21.866416 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:21.866460 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:21.866939 (+ 479us) server_negotiation.cc:407] Connection header received
0808 10:34:21.867280 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:21.867287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:21.867524 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:21.867730 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:21.868771 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.870431 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.872785 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:21.874062 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:21.874285 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:21.874474 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:21.875412 (+ 938us) server_negotiation.cc:299] Negotiation successful
0808 10:34:21.875760 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":792,"thread_start_us":149,"threads_started":1}
I20260808 10:34:21.878945 32060 tablet_service.cc:1467] Processing CreateTablet for tablet be8d557cd8f240fcaee182074e220d8d (DEFAULT_TABLE table=test-table [id=60a9b8d70258478394fb3773b6c25f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:21.878978 32204 tablet_service.cc:1467] Processing CreateTablet for tablet be8d557cd8f240fcaee182074e220d8d (DEFAULT_TABLE table=test-table [id=60a9b8d70258478394fb3773b6c25f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:21.880049 32060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be8d557cd8f240fcaee182074e220d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:21.880198 32204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be8d557cd8f240fcaee182074e220d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:21.881165 32348 tablet_service.cc:1467] Processing CreateTablet for tablet be8d557cd8f240fcaee182074e220d8d (DEFAULT_TABLE table=test-table [id=60a9b8d70258478394fb3773b6c25f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:21.882345 32348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be8d557cd8f240fcaee182074e220d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:21.888657 32440 tablet_bootstrap.cc:492] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba: Bootstrap starting.
I20260808 10:34:21.890558 32441 tablet_bootstrap.cc:492] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1: Bootstrap starting.
I20260808 10:34:21.892359 32440 tablet_bootstrap.cc:654] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:21.892472 32442 tablet_bootstrap.cc:492] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73: Bootstrap starting.
I20260808 10:34:21.894269 32441 tablet_bootstrap.cc:654] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:21.894688 32440 log.cc:824] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:21.896853 32441 log.cc:824] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:21.896917 32442 tablet_bootstrap.cc:654] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:21.898444 32440 tablet_bootstrap.cc:492] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba: No bootstrap required, opened a new log
I20260808 10:34:21.898590 32442 log.cc:824] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:21.898710 32440 ts_tablet_manager.cc:1397] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 10:34:21.899811 32441 tablet_bootstrap.cc:492] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1: No bootstrap required, opened a new log
I20260808 10:34:21.900077 32441 ts_tablet_manager.cc:1397] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 10:34:21.901118 32442 tablet_bootstrap.cc:492] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73: No bootstrap required, opened a new log
I20260808 10:34:21.901451 32442 ts_tablet_manager.cc:1397] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260808 10:34:21.906749 32442 raft_consensus.cc:348] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0fc54654439470cb4dbc394b81d54ba" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 42183 } } peers { permanent_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 44159 } } peers { permanent_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 45321 } }
I20260808 10:34:21.907222 32442 raft_consensus.cc:374] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:21.907346 32442 raft_consensus.cc:729] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd0f5b9717f4e90a5167ec5ad0c5d73, State: Initialized, Role: FOLLOWER
I20260808 10:34:21.908027 32442 consensus_queue.cc:260] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0fc54654439470cb4dbc394b81d54ba" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 42183 } } peers { permanent_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 44159 } } peers { permanent_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 45321 } }
I20260808 10:34:21.907810 32441 raft_consensus.cc:348] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0fc54654439470cb4dbc394b81d54ba" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 42183 } } peers { permanent_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 44159 } } peers { permanent_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 45321 } }
I20260808 10:34:21.908309 32441 raft_consensus.cc:374] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:21.908422 32441 raft_consensus.cc:729] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe81a83fc0a4cb1be178b71fa01d3a1, State: Initialized, Role: FOLLOWER
I20260808 10:34:21.908396 32440 raft_consensus.cc:348] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0fc54654439470cb4dbc394b81d54ba" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 42183 } } peers { permanent_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 44159 } } peers { permanent_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 45321 } }
I20260808 10:34:21.908772 32440 raft_consensus.cc:374] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:21.908839 32440 raft_consensus.cc:729] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0fc54654439470cb4dbc394b81d54ba, State: Initialized, Role: FOLLOWER
I20260808 10:34:21.909108 32441 consensus_queue.cc:260] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0fc54654439470cb4dbc394b81d54ba" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 42183 } } peers { permanent_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 44159 } } peers { permanent_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 45321 } }
I20260808 10:34:21.909359 32421 heartbeater.cc:503] Master 127.30.146.190:32971 was elected leader, sending a full tablet report...
I20260808 10:34:21.909485 32440 consensus_queue.cc:260] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0fc54654439470cb4dbc394b81d54ba" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 42183 } } peers { permanent_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 44159 } } peers { permanent_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 45321 } }
I20260808 10:34:21.910308 32275 heartbeater.cc:503] Master 127.30.146.190:32971 was elected leader, sending a full tablet report...
I20260808 10:34:21.910672 32441 ts_tablet_manager.cc:1428] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260808 10:34:21.910851 32440 ts_tablet_manager.cc:1428] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260808 10:34:21.911317 32133 heartbeater.cc:503] Master 127.30.146.190:32971 was elected leader, sending a full tablet report...
I20260808 10:34:21.913612 32442 ts_tablet_manager.cc:1428] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
W20260808 10:34:21.945369 32276 tablet.cc:2369] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:34:22.034963 32422 tablet.cc:2369] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:34:22.046185 32450 raft_consensus.cc:482] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:22.046598 32450 raft_consensus.cc:504] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0fc54654439470cb4dbc394b81d54ba" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 42183 } } peers { permanent_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 44159 } } peers { permanent_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 45321 } }
I20260808 10:34:22.048069 32450 leader_election.cc:288] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dd0f5b9717f4e90a5167ec5ad0c5d73 (127.30.146.131:44159), 8fe81a83fc0a4cb1be178b71fa01d3a1 (127.30.146.130:45321)
I20260808 10:34:22.053573 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:22.048641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:56565 (local address 127.30.146.130:45321)
0808 10:34:22.048846 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:22.048854 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:22.049351 (+ 497us) server_negotiation.cc:407] Connection header received
0808 10:34:22.049529 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:22.049533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:22.049596 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:22.049723 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:22.050270 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.050813 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.052275 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.052775 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.052913 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:22.052987 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:22.053323 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 10:34:22.053399 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:34:22.053742 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:22.048637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:45321 (local address 127.30.146.129:56565)
0808 10:34:22.049212 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:22.049232 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:22.049389 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:22.049744 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:22.049747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:22.049776 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:22.050122 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:22.050130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.050955 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.050970 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:22.052096 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:22.052115 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.052911 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.052915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:22.053060 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:22.053089 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 10:34:22.053298 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 10:34:22.053379 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":1,"spinlock_wait_cycles":2816,"thread_start_us":190,"threads_started":1}
I20260808 10:34:22.054455 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:22.048825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:33477 (local address 127.30.146.131:44159)
0808 10:34:22.049002 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:22.049008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:22.049862 (+ 854us) server_negotiation.cc:407] Connection header received
0808 10:34:22.050105 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:22.050109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:22.050173 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:22.050262 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:22.050585 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.051164 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.052747 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.053380 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.053535 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:22.053629 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:22.054242 (+ 613us) server_negotiation.cc:299] Negotiation successful
0808 10:34:22.054364 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:34:22.054544 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:22.048637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:44159 (local address 127.30.146.129:33477)
0808 10:34:22.049684 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:22.049703 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:22.049942 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:22.050270 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:22.050274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:22.050290 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:22.050461 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:22.050467 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.051351 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.051354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:22.052547 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:22.052558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.053517 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.053532 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:22.053779 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:22.053834 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 10:34:22.054195 (+ 361us) client_negotiation.cc:240] Negotiation successful
0808 10:34:22.054338 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":232,"thread_start_us":419,"threads_started":1}
I20260808 10:34:22.054786 32227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be8d557cd8f240fcaee182074e220d8d" candidate_uuid: "e0fc54654439470cb4dbc394b81d54ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" is_pre_election: true
I20260808 10:34:22.055308 32227 raft_consensus.cc:2454] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0fc54654439470cb4dbc394b81d54ba in term 0.
I20260808 10:34:22.055356 32370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be8d557cd8f240fcaee182074e220d8d" candidate_uuid: "e0fc54654439470cb4dbc394b81d54ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" is_pre_election: true
I20260808 10:34:22.055657 32370 raft_consensus.cc:2454] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0fc54654439470cb4dbc394b81d54ba in term 0.
I20260808 10:34:22.056002 32012 leader_election.cc:302] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe81a83fc0a4cb1be178b71fa01d3a1, e0fc54654439470cb4dbc394b81d54ba; no voters:
I20260808 10:34:22.056435 32450 raft_consensus.cc:2781] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:22.056563 32450 raft_consensus.cc:482] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:22.056679 32450 raft_consensus.cc:3037] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:22.058435 32450 raft_consensus.cc:504] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0fc54654439470cb4dbc394b81d54ba" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 42183 } } peers { permanent_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 44159 } } peers { permanent_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 45321 } }
I20260808 10:34:22.058929 32450 leader_election.cc:288] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [CANDIDATE]: Term 1 election: Requested vote from peers 1dd0f5b9717f4e90a5167ec5ad0c5d73 (127.30.146.131:44159), 8fe81a83fc0a4cb1be178b71fa01d3a1 (127.30.146.130:45321)
I20260808 10:34:22.059322 32370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be8d557cd8f240fcaee182074e220d8d" candidate_uuid: "e0fc54654439470cb4dbc394b81d54ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73"
I20260808 10:34:22.059331 32227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be8d557cd8f240fcaee182074e220d8d" candidate_uuid: "e0fc54654439470cb4dbc394b81d54ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1"
I20260808 10:34:22.059458 32370 raft_consensus.cc:3037] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:22.059469 32227 raft_consensus.cc:3037] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:22.061000 32227 raft_consensus.cc:2454] T be8d557cd8f240fcaee182074e220d8d P 8fe81a83fc0a4cb1be178b71fa01d3a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0fc54654439470cb4dbc394b81d54ba in term 1.
I20260808 10:34:22.061190 32370 raft_consensus.cc:2454] T be8d557cd8f240fcaee182074e220d8d P 1dd0f5b9717f4e90a5167ec5ad0c5d73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0fc54654439470cb4dbc394b81d54ba in term 1.
I20260808 10:34:22.061410 32012 leader_election.cc:302] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe81a83fc0a4cb1be178b71fa01d3a1, e0fc54654439470cb4dbc394b81d54ba; no voters:
I20260808 10:34:22.061709 32450 raft_consensus.cc:2781] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:22.062001 32450 raft_consensus.cc:686] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [term 1 LEADER]: Becoming Leader. State: Replica: e0fc54654439470cb4dbc394b81d54ba, State: Running, Role: LEADER
I20260808 10:34:22.062397 32450 consensus_queue.cc:237] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0fc54654439470cb4dbc394b81d54ba" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 42183 } } peers { permanent_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 44159 } } peers { permanent_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 45321 } }
I20260808 10:34:22.066524 31928 catalog_manager.cc:5495] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba reported cstate change: term changed from 0 to 1, leader changed from <none> to e0fc54654439470cb4dbc394b81d54ba (127.30.146.129). New cstate: current_term: 1 leader_uuid: "e0fc54654439470cb4dbc394b81d54ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0fc54654439470cb4dbc394b81d54ba" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 42183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dd0f5b9717f4e90a5167ec5ad0c5d73" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 44159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe81a83fc0a4cb1be178b71fa01d3a1" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 45321 } health_report { overall_health: UNKNOWN } } }
Aug 08 10:34:22 dist-test-slave-9gcw krb5kdc[31878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:34:22.116605 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:22.104807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33534 (local address 127.30.146.129:42183)
0808 10:34:22.105050 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:22.105056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:22.105340 (+ 284us) server_negotiation.cc:407] Connection header received
0808 10:34:22.105574 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:22.105583 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:22.105630 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:22.105720 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:22.106831 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.107384 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.108606 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.108837 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.108949 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:22.112236 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:22.112251 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:22.112272 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:22.112299 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:22.114724 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:22.115224 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:22.115230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:22.115231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:22.115337 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:22.115597 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:22.115600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:22.115601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:22.115865 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:22.116070 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:22.116247 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 10:34:22.116368 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 10:34:22 dist-test-slave-9gcw krb5kdc[31878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185262, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:34:22 dist-test-slave-9gcw krb5kdc[31878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185262, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
W20260808 10:34:22.149430 32134 tablet.cc:2369] T be8d557cd8f240fcaee182074e220d8d P e0fc54654439470cb4dbc394b81d54ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:34:22.151784 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:22.141528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33542 (local address 127.30.146.129:42183)
0808 10:34:22.141723 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:22.141734 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:22.141830 (+ 96us) server_negotiation.cc:407] Connection header received
0808 10:34:22.141950 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:22.141952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:22.141997 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:22.142083 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:22.142875 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.143383 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.144157 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.144465 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.144549 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:22.147683 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:22.147687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:22.147690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:22.147719 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:22.149581 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:22.150211 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:22.150214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:22.150216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:22.150305 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:22.150673 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:22.150676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:22.150678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:22.150888 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:22.151033 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:22.151476 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 10:34:22.151607 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
Aug 08 10:34:22 dist-test-slave-9gcw krb5kdc[31878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:34:22 dist-test-slave-9gcw krb5kdc[31878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:34:22.190052 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:22.180919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33556 (local address 127.30.146.129:42183)
0808 10:34:22.181098 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:22.181102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:22.181405 (+ 303us) server_negotiation.cc:407] Connection header received
0808 10:34:22.181563 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:22.181566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:22.181622 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:22.181704 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:22.182472 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.182975 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.183786 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:22.183988 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:22.184063 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:22.187035 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:22.187038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:22.187040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:22.187063 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:22.188325 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:22.188848 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:22.188850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:22.188852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:22.188945 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:22.189281 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:22.189293 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:22.189294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:22.189492 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:22.189595 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:22.189830 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 10:34:22.189926 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 10:34:22.191614 32037 tablet_service.cc:2908] Rejecting scan request for tablet be8d557cd8f240fcaee182074e220d8d: Uninitialized: clean time has not yet been initialized
I20260808 10:34:22.194288 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 31993
I20260808 10:34:22.203363 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 32138
I20260808 10:34:22.211807 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 32281
I20260808 10:34:22.219476 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 31891
2026-08-08T10:34:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2925 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:22 dist-test-slave-9gcw krb5kdc[32467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:22 dist-test-slave-9gcw krb5kdc[32467](info): set up 2 sockets
Aug 08 10:34:22 dist-test-slave-9gcw krb5kdc[32467](info): commencing operation
krb5kdc: starting...
W20260808 10:34:23.481879 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:23 dist-test-slave-9gcw krb5kdc[32467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:34:23.637495 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:39571
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:39571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:23.776471 32481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:23.776839 32481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:23.776917 32481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:23.781888 32481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:23.782008 32481 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:34:23.782039 32481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:23.782078 32481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:23.782109 32481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:23.782148 32481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:23.787938 32481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:39571
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:39571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:23.789299 32481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:23.790417 32481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:23.800463 32490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:23.800434 32488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:23.800561 32481 server_base.cc:1034] running on GCE node
W20260808 10:34:23.800467 32487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:23.801520 32481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:23.803042 32481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:23.804240 32481 hybrid_clock.cc:648] HybridClock initialized: now 1786185263804200 us; error 54 us; skew 500 ppm
Aug 08 10:34:23 dist-test-slave-9gcw krb5kdc[32467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:23.807804 32481 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:34:23.808874 32481 webserver.cc:460] Webserver started at http://127.30.146.190:44465/ using document root <none> and password file <none>
I20260808 10:34:23.809674 32481 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:23.809791 32481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:23.810093 32481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:23.812448 32481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "84a8357ae09f4262a6d6250c4020b120"
format_stamp: "Formatted at 2026-08-08 10:34:23 on dist-test-slave-9gcw"
server_key: "28f90ee1a8c949310a6fb56c94839fc3"
server_key_iv: "1b0dcbf62f40dd5f052e1abf8310e477"
server_key_version: "encryptionkey@0"
I20260808 10:34:23.813220 32481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "84a8357ae09f4262a6d6250c4020b120"
format_stamp: "Formatted at 2026-08-08 10:34:23 on dist-test-slave-9gcw"
server_key: "28f90ee1a8c949310a6fb56c94839fc3"
server_key_iv: "1b0dcbf62f40dd5f052e1abf8310e477"
server_key_version: "encryptionkey@0"
I20260808 10:34:23.817546 32481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:34:23.820428 32498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:23.821899 32481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:23.822088 32481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "84a8357ae09f4262a6d6250c4020b120"
format_stamp: "Formatted at 2026-08-08 10:34:23 on dist-test-slave-9gcw"
server_key: "28f90ee1a8c949310a6fb56c94839fc3"
server_key_iv: "1b0dcbf62f40dd5f052e1abf8310e477"
server_key_version: "encryptionkey@0"
I20260808 10:34:23.822220 32481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:23.843082 32481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:23.847373 32481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:23.847621 32481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:23.857666 32481 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:39571
I20260808 10:34:23.857723 32563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:39571 every 8 connection(s)
I20260808 10:34:23.858795 32481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:23.862146 32564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:23.866716 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 32481
I20260808 10:34:23.866865 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:23.867179 31306 external_mini_cluster.cc:1428] Setting key 02d324cb82e3631b20459f46bea9b5e9
I20260808 10:34:23.868788 32564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120: Bootstrap starting.
I20260808 10:34:23.871838 32564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:23.872759 32564 log.cc:824] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:23.874938 32564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120: No bootstrap required, opened a new log
Aug 08 10:34:23 dist-test-slave-9gcw krb5kdc[32467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:23.879750 32564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a8357ae09f4262a6d6250c4020b120" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39571 } }
I20260808 10:34:23.880091 32564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:23.880187 32564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84a8357ae09f4262a6d6250c4020b120, State: Initialized, Role: FOLLOWER
I20260808 10:34:23.880857 32564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a8357ae09f4262a6d6250c4020b120" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39571 } }
I20260808 10:34:23.881078 32564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:23.881202 32564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:23.881333 32564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:23.882452 32564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a8357ae09f4262a6d6250c4020b120" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39571 } }
I20260808 10:34:23.882866 32564 leader_election.cc:302] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84a8357ae09f4262a6d6250c4020b120; no voters:
I20260808 10:34:23.883257 32564 leader_election.cc:288] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:23.883344 32570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:23.883527 32570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [term 1 LEADER]: Becoming Leader. State: Replica: 84a8357ae09f4262a6d6250c4020b120, State: Running, Role: LEADER
I20260808 10:34:23.883827 32570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a8357ae09f4262a6d6250c4020b120" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39571 } }
I20260808 10:34:23.884238 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:23.868665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.30.146.190:39571)
0808 10:34:23.869135 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:23.869201 (+ 66us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:23.869251 (+ 50us) server_negotiation.cc:407] Connection header received
0808 10:34:23.869925 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:23.869943 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:23.870428 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:23.870799 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:23.872041 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:23.873292 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:23.874351 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:23.874836 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:23.875020 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:23.878198 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:23.878203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:23.878218 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:23.878248 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:23.880377 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:23.880996 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:23.881000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:23.881005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:23.881231 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:23.881585 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:23.881588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:23.881590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:23.882140 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:23.882372 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:23.882712 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 10:34:23.883098 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":157,"threads_started":1}
I20260808 10:34:23.884593 32564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:23.885561 32573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84a8357ae09f4262a6d6250c4020b120. Latest consensus state: current_term: 1 leader_uuid: "84a8357ae09f4262a6d6250c4020b120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a8357ae09f4262a6d6250c4020b120" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39571 } } }
I20260808 10:34:23.885705 32573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:23.886013 32572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84a8357ae09f4262a6d6250c4020b120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a8357ae09f4262a6d6250c4020b120" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39571 } } }
I20260808 10:34:23.886118 32572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:23.886691 32575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:23.889690 32575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:23.897104 32575 catalog_manager.cc:1347] Generated new cluster ID: 27245f5c181f433784e5f316fc5079b2
I20260808 10:34:23.897240 32575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:23.919232 32575 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:23.920306 32575 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:23.930020 32575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120: Generated new TSK 0
I20260808 10:34:23.930763 32575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:34:23.969811 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39571
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:24.107749 32593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:24.108001 32593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:24.108045 32593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:24.112262 32593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:24.112336 32593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:24.112375 32593 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:34:24.112461 32593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:24.117858 32593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.146.190:39571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:24.119035 32593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:24.120193 32593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:24.131399 32603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:24.131454 32600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:24.131659 32593 server_base.cc:1034] running on GCE node
W20260808 10:34:24.131428 32601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:24.132449 32593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:24.133249 32593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:24.134464 32593 hybrid_clock.cc:648] HybridClock initialized: now 1786185264134441 us; error 37 us; skew 500 ppm
Aug 08 10:34:24 dist-test-slave-9gcw krb5kdc[32467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:24.137982 32593 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:34:24.139401 32593 webserver.cc:460] Webserver started at http://127.30.146.129:41015/ using document root <none> and password file <none>
I20260808 10:34:24.140153 32593 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:24.140264 32593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:24.140519 32593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:24.142890 32593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "3a2413a1d86d4b639b1f2d074d6672fc"
format_stamp: "Formatted at 2026-08-08 10:34:24 on dist-test-slave-9gcw"
server_key: "b78c6b5ce3f9c0ce5bdb9da964de9e73"
server_key_iv: "03435f06c602d3fc5fc2254603e3d05e"
server_key_version: "encryptionkey@0"
I20260808 10:34:24.143558 32593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "3a2413a1d86d4b639b1f2d074d6672fc"
format_stamp: "Formatted at 2026-08-08 10:34:24 on dist-test-slave-9gcw"
server_key: "b78c6b5ce3f9c0ce5bdb9da964de9e73"
server_key_iv: "03435f06c602d3fc5fc2254603e3d05e"
server_key_version: "encryptionkey@0"
I20260808 10:34:24.148108 32593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:34:24.151062 32610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:24.152631 32593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:24.152822 32593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "3a2413a1d86d4b639b1f2d074d6672fc"
format_stamp: "Formatted at 2026-08-08 10:34:24 on dist-test-slave-9gcw"
server_key: "b78c6b5ce3f9c0ce5bdb9da964de9e73"
server_key_iv: "03435f06c602d3fc5fc2254603e3d05e"
server_key_version: "encryptionkey@0"
I20260808 10:34:24.152973 32593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:24.172851 32593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:24.176906 32593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:24.177206 32593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:24.178874 32593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:24.178949 32593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:24.179019 32593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:24.179075 32593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:34:24 dist-test-slave-9gcw krb5kdc[32467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:24.192164 32593 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:36925
I20260808 10:34:24.192214 32733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:36925 every 8 connection(s)
I20260808 10:34:24.193375 32593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:24.195379 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.179796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:37079 (local address 127.30.146.190:39571)
0808 10:34:24.179976 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.179983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.180824 (+ 841us) server_negotiation.cc:407] Connection header received
0808 10:34:24.181947 (+ 1123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:24.181953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:24.182017 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:24.182083 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:24.184101 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.185058 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.186769 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.187067 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.187242 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.190831 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:24.190836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:24.190839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:24.190873 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:24.192881 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.193531 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.193535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.193537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.193659 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.194150 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.194153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.194155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.194507 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:24.194625 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:24.195110 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 10:34:24.195224 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:34:24.196302 32619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.180183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39571 (local address 127.30.146.129:37079)
0808 10:34:24.180642 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:24.180694 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:24.181684 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:24.182600 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:24.182611 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:24.183091 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:24.183910 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.183929 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.185231 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.185261 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.186632 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.186642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.187324 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.187330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.187520 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.188211 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:24.188234 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:24.190662 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:24.193017 (+ 2355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.193023 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.193038 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.193419 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.193823 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.193826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.193828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.194032 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.194639 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:24.194644 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:24.194870 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 10:34:24.195035 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 10:34:24.195340 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":136,"threads_started":1}
I20260808 10:34:24.198100 32734 heartbeater.cc:347] Connected to a master server at 127.30.146.190:39571
I20260808 10:34:24.198298 32734 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:24.198889 32734 heartbeater.cc:511] Master 127.30.146.190:39571 requested a full tablet report, sending...
I20260808 10:34:24.200632 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 32593
I20260808 10:34:24.200759 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:24.200733 32514 ts_manager.cc:194] Registered new tserver with Master: 3a2413a1d86d4b639b1f2d074d6672fc (127.30.146.129:36925)
I20260808 10:34:24.201042 31306 external_mini_cluster.cc:1428] Setting key 9da64176c9d3eae471f1b7834ef4b459
I20260808 10:34:24.202225 32514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:37079
I20260808 10:34:24.210836 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.200627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:42525 (local address 127.30.146.190:39571)
0808 10:34:24.200854 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.200859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.201751 (+ 892us) server_negotiation.cc:407] Connection header received
0808 10:34:24.201894 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:24.201897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:24.201942 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:24.202026 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:24.202581 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.203326 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.205657 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.205888 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.206006 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.207315 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:24.207318 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:24.207320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:24.207342 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:24.209024 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.209624 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.209627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.209628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.209703 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.210028 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.210030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.210031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.210192 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:24.210264 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:24.210602 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 10:34:24.210700 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260808 10:34:24.210924 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.200543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39571 (local address 127.30.146.129:42525)
0808 10:34:24.201660 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:24.201673 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:24.201792 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:24.202070 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:24.202073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:24.202293 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:24.202451 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.202459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.204875 (+ 2416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.204878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.205524 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.205534 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.206016 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.206020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.206129 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.206642 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:24.206662 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:24.207174 (+ 512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:24.209188 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.209195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.209199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.209504 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.209805 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.209809 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.209811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.209921 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.210298 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:24.210303 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:24.210426 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 10:34:24.210533 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 10:34:24.210689 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:34:24.232853 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39571
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:24.356030 32743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:24.356323 32743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:24.356374 32743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:24.361027 32743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:24.361102 32743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:24.361207 32743 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:34:24.361253 32743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:24.367198 32743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.146.190:39571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:24.368417 32743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:24.369741 32743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:24.380424 32749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:24.380465 32753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:24.380409 32750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:24.380940 32743 server_base.cc:1034] running on GCE node
I20260808 10:34:24.381500 32743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:24.382841 32743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:24.384167 32743 hybrid_clock.cc:648] HybridClock initialized: now 1786185264384116 us; error 69 us; skew 500 ppm
Aug 08 10:34:24 dist-test-slave-9gcw krb5kdc[32467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:24.387499 32743 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:34:24.388540 32743 webserver.cc:460] Webserver started at http://127.30.146.130:33389/ using document root <none> and password file <none>
I20260808 10:34:24.389207 32743 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:24.389308 32743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:24.389556 32743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:24.391984 32743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "186d7aca502e493a885b79af4a80dcc8"
format_stamp: "Formatted at 2026-08-08 10:34:24 on dist-test-slave-9gcw"
server_key: "b9d349d6a8dd7f2353813270a8a89c4c"
server_key_iv: "21f854b6b1ddb832ae02d3619c1ce670"
server_key_version: "encryptionkey@0"
I20260808 10:34:24.392612 32743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "186d7aca502e493a885b79af4a80dcc8"
format_stamp: "Formatted at 2026-08-08 10:34:24 on dist-test-slave-9gcw"
server_key: "b9d349d6a8dd7f2353813270a8a89c4c"
server_key_iv: "21f854b6b1ddb832ae02d3619c1ce670"
server_key_version: "encryptionkey@0"
I20260808 10:34:24.397765 32743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 10:34:24.400677 32760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:24.402084 32743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:24.402244 32743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "186d7aca502e493a885b79af4a80dcc8"
format_stamp: "Formatted at 2026-08-08 10:34:24 on dist-test-slave-9gcw"
server_key: "b9d349d6a8dd7f2353813270a8a89c4c"
server_key_iv: "21f854b6b1ddb832ae02d3619c1ce670"
server_key_version: "encryptionkey@0"
I20260808 10:34:24.402390 32743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:24.422745 32743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:24.427444 32743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:24.427670 32743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:24.429605 32743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:24.429709 32743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:24.429767 32743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:24.429801 32743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:24.441850 32743 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:36369
I20260808 10:34:24.441906 418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:36369 every 8 connection(s)
Aug 08 10:34:24 dist-test-slave-9gcw krb5kdc[32467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:24.442956 32743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:24.447229 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.430469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:59531 (local address 127.30.146.190:39571)
0808 10:34:24.430607 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.430611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.431911 (+ 1300us) server_negotiation.cc:407] Connection header received
0808 10:34:24.433255 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:24.433259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:24.433362 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:24.433428 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:24.435584 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.436581 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.438619 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.439046 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.439199 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.442693 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:24.442697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:24.442699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:24.442734 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:24.444789 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.445491 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.445494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.445495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.445582 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.445910 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.445913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.445914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.446124 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:24.446203 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:24.446727 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 10:34:24.446865 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:34:24.447906 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.431118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39571 (local address 127.30.146.130:59531)
0808 10:34:24.431736 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:24.431799 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:24.432859 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:24.433963 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:24.433972 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:24.434538 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:24.435380 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.435401 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.436747 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.436751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.438471 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.438482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.439290 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.439296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.439547 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.440379 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:24.440404 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:24.442471 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:24.444950 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.444954 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.444966 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.445374 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.445677 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.445679 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.445681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.445816 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.446218 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:24.446223 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:24.446471 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 10:34:24.446638 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 10:34:24.446968 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":176,"threads_started":1}
I20260808 10:34:24.449348 419 heartbeater.cc:347] Connected to a master server at 127.30.146.190:39571
I20260808 10:34:24.449527 419 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:24.450137 419 heartbeater.cc:511] Master 127.30.146.190:39571 requested a full tablet report, sending...
I20260808 10:34:24.451329 32514 ts_manager.cc:194] Registered new tserver with Master: 186d7aca502e493a885b79af4a80dcc8 (127.30.146.130:36369)
I20260808 10:34:24.451349 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 32743
I20260808 10:34:24.451478 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:24.451792 31306 external_mini_cluster.cc:1428] Setting key 93f963fc82f7550979ab185a8282b666
I20260808 10:34:24.452088 32514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:59531
I20260808 10:34:24.462304 425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.451867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39571 (local address 127.30.146.130:39809)
0808 10:34:24.452228 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:24.452243 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:24.452407 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:24.452767 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:24.452770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:24.453040 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:24.453206 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.453215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.454249 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.454256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.455453 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.455468 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.455992 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.455998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.456122 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.456851 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:24.456878 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:24.457595 (+ 717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:24.460045 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.460048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.460050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.460376 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.460721 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.460723 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.460725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.460811 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.461618 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:24.461621 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:24.461823 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 10:34:24.461968 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 10:34:24.462125 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":104,"threads_started":1}
I20260808 10:34:24.462564 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.451949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:39809 (local address 127.30.146.190:39571)
0808 10:34:24.452056 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.452061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.452326 (+ 265us) server_negotiation.cc:407] Connection header received
0808 10:34:24.452574 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:24.452577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:24.452630 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:24.452714 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:24.453345 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.454085 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.455603 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.455840 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.455929 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.457806 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:24.457813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:24.457818 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:24.457855 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:24.459893 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.460507 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.460511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.460512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.460613 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.460923 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.460929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.460934 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.461414 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:24.461556 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:24.462076 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 10:34:24.462276 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:34:24.489360 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39571
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:24.630606 429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:24.630946 429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:24.631052 429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:24.635501 429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:24.635615 429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:24.635689 429 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:34:24.635774 429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:24.641628 429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.146.190:39571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:24.642865 429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:24.643980 429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:24.655653 435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:24.655658 437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:24.655658 434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:24.656039 429 server_base.cc:1034] running on GCE node
I20260808 10:34:24.656623 429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:24.657624 429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:24.658883 429 hybrid_clock.cc:648] HybridClock initialized: now 1786185264658844 us; error 51 us; skew 500 ppm
Aug 08 10:34:24 dist-test-slave-9gcw krb5kdc[32467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:24.662318 429 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:34:24.663281 429 webserver.cc:460] Webserver started at http://127.30.146.131:37737/ using document root <none> and password file <none>
I20260808 10:34:24.663844 429 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:24.663905 429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:24.664103 429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:24.666504 429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "54638c91e82f4d1c9243c8137a3b0155"
format_stamp: "Formatted at 2026-08-08 10:34:24 on dist-test-slave-9gcw"
server_key: "4fe289afe3159656de592332ee15d578"
server_key_iv: "cd56c0b95bd9d4025f0fc6457f3c4f77"
server_key_version: "encryptionkey@0"
I20260808 10:34:24.667167 429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "54638c91e82f4d1c9243c8137a3b0155"
format_stamp: "Formatted at 2026-08-08 10:34:24 on dist-test-slave-9gcw"
server_key: "4fe289afe3159656de592332ee15d578"
server_key_iv: "cd56c0b95bd9d4025f0fc6457f3c4f77"
server_key_version: "encryptionkey@0"
I20260808 10:34:24.672103 429 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 10:34:24.675171 444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:24.676472 429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:24.676680 429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "54638c91e82f4d1c9243c8137a3b0155"
format_stamp: "Formatted at 2026-08-08 10:34:24 on dist-test-slave-9gcw"
server_key: "4fe289afe3159656de592332ee15d578"
server_key_iv: "cd56c0b95bd9d4025f0fc6457f3c4f77"
server_key_version: "encryptionkey@0"
I20260808 10:34:24.676824 429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:24.689324 429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:24.692900 429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:24.693128 429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:24.694916 429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:24.694970 429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:24.695012 429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:24.695071 429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:34:24 dist-test-slave-9gcw krb5kdc[32467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:24.707439 429 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:42797
I20260808 10:34:24.707926 569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:42797 every 8 connection(s)
I20260808 10:34:24.708555 429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:24.710872 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.695932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:45329 (local address 127.30.146.190:39571)
0808 10:34:24.696076 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.696080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.697344 (+ 1264us) server_negotiation.cc:407] Connection header received
0808 10:34:24.698638 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:24.698642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:24.698706 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:24.698828 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:24.700601 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.701585 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.702969 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.703211 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.703380 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.706602 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:24.706609 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:24.706616 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:24.706651 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:24.708611 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.709340 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.709344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.709346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.709443 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.709872 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.709874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.709876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.710075 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:24.710170 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:24.710637 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 10:34:24.710744 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:34:24.711869 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.696570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39571 (local address 127.30.146.131:45329)
0808 10:34:24.697129 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:24.697229 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:24.698264 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:24.699056 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:24.699066 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:24.699574 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:24.700366 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.700393 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.701744 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.701748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.702827 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.702854 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.703432 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.703436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.703619 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.704326 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:24.704347 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:24.706400 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:24.708777 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.708782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.708797 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.709200 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.709578 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.709581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.709582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.709758 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.710184 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:24.710189 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:24.710418 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 10:34:24.710565 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 10:34:24.710852 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":183,"threads_started":1}
I20260808 10:34:24.713196 570 heartbeater.cc:347] Connected to a master server at 127.30.146.190:39571
I20260808 10:34:24.713407 570 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:24.713951 570 heartbeater.cc:511] Master 127.30.146.190:39571 requested a full tablet report, sending...
I20260808 10:34:24.715233 32515 ts_manager.cc:194] Registered new tserver with Master: 54638c91e82f4d1c9243c8137a3b0155 (127.30.146.131:42797)
I20260808 10:34:24.716077 32515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:45329
I20260808 10:34:24.718227 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 429
I20260808 10:34:24.718365 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:24.718640 31306 external_mini_cluster.cc:1428] Setting key 65c8a385c93fbc7cf4730918c43fff52
I20260808 10:34:24.721302 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:34:24.723838 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.715748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:57779 (local address 127.30.146.190:39571)
0808 10:34:24.715890 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.715895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.716065 (+ 170us) server_negotiation.cc:407] Connection header received
0808 10:34:24.716161 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:24.716164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:24.716204 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:24.716279 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:24.716771 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.717601 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.718831 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.719043 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.719170 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.720246 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:24.720250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:24.720253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:24.720283 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:24.721998 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.722534 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.722537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.722538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.722620 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.722930 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.722932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.722933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.723137 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:24.723218 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:24.723567 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 10:34:24.723669 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:34:24.724226 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.715619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39571 (local address 127.30.146.131:57779)
0808 10:34:24.715977 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:24.715991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:24.716077 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:24.716293 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:24.716295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:24.716555 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:24.716657 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.716663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.717748 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.717751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.718663 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.718677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.719176 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.719180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.719275 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.719768 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:24.719780 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:24.720100 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:24.722133 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.722139 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.722144 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.722427 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.722718 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:24.722724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:24.722728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:24.722836 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:24.723241 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:24.723246 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:24.723392 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 10:34:24.723512 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 10:34:24.723725 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":119,"threads_started":1}
Aug 08 10:34:24 dist-test-slave-9gcw krb5kdc[32467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185264, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:34:24 dist-test-slave-9gcw krb5kdc[32467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185264, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:24.751612 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.742397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.30.146.190:39571)
0808 10:34:24.742530 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.742534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.742546 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:34:24.742587 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:24.742589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:24.742629 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:24.742730 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:24.743487 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.744047 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.744980 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.745281 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.745399 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.747987 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:24.747991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:24.747994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:24.748028 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:24.749726 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.750280 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.750283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.750285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.750375 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:24.750673 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:24.750678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:24.750679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:24.750903 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:24.751058 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:24.751315 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 10:34:24.751439 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:34:24.755055 32515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:34:24.758605 32515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:24.774802 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.768027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.30.146.131:42797)
0808 10:34:24.768777 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.768790 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.769230 (+ 440us) server_negotiation.cc:407] Connection header received
0808 10:34:24.769476 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:24.769482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:24.769683 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:24.769806 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:24.770278 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.771477 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.773544 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.774081 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.774252 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.774403 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:24.774558 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:34:24.774651 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":109,"threads_started":1}
I20260808 10:34:24.774792 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.767893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:42797 (local address 127.0.0.1:52948)
0808 10:34:24.769064 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:24.769116 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:24.769318 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:24.769978 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:24.769980 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:24.770006 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:24.770148 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.770155 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.771622 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.771627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.773382 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.773394 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.774234 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.774240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.774354 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.774392 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:34:24.774509 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 10:34:24.774637 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":108,"spinlock_wait_cycles":5888,"thread_start_us":111,"threads_started":1}
I20260808 10:34:24.775118 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.769059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42136 (local address 127.30.146.130:36369)
0808 10:34:24.769519 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.769531 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.769554 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:34:24.769635 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:24.769647 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:24.769841 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:24.769989 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:24.771111 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.772090 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.773307 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.774105 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.774248 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.774406 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:24.774859 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 10:34:24.774964 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":148,"threads_started":1}
I20260808 10:34:24.775496 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.768634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:36369 (local address 127.0.0.1:42136)
0808 10:34:24.769409 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:24.769425 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:24.769519 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:24.769971 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:24.769974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:24.770000 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:24.770153 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.770158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.772216 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.772221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.773114 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.773124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.774467 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.774471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.774579 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.774601 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:34:24.774674 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:34:24.774756 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":56,"spinlock_wait_cycles":9344,"thread_start_us":92,"threads_started":1}
I20260808 10:34:24.775945 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.768336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36925 (local address 127.0.0.1:34026)
0808 10:34:24.769069 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:24.769116 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:24.769279 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:24.769849 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:24.769857 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:24.769879 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:24.770037 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.770045 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.771631 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.771634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.773387 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:24.773394 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.775398 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.775401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:24.775492 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.775515 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:24.775719 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 10:34:24.775824 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":237,"thread_start_us":96,"threads_started":1}
I20260808 10:34:24.775969 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.768677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34026 (local address 127.30.146.129:36925)
0808 10:34:24.769192 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.769201 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.769229 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:34:24.769476 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:24.769482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:24.769683 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:24.769826 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:24.770183 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.771477 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.773517 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:24.775264 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:24.775419 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:24.775540 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:24.775715 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:34:24.775843 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":133,"threads_started":1}
I20260808 10:34:24.778261 500 tablet_service.cc:1467] Processing CreateTablet for tablet 8d2fb70d92424afcb57f075925a671f9 (DEFAULT_TABLE table=test-table [id=07b4e0b575a74ee3844e280fa01d991b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:24.778244 346 tablet_service.cc:1467] Processing CreateTablet for tablet 8d2fb70d92424afcb57f075925a671f9 (DEFAULT_TABLE table=test-table [id=07b4e0b575a74ee3844e280fa01d991b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:24.779369 346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d2fb70d92424afcb57f075925a671f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:24.778978 32663 tablet_service.cc:1467] Processing CreateTablet for tablet 8d2fb70d92424afcb57f075925a671f9 (DEFAULT_TABLE table=test-table [id=07b4e0b575a74ee3844e280fa01d991b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:24.779404 500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d2fb70d92424afcb57f075925a671f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:24.779933 32663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d2fb70d92424afcb57f075925a671f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:24.786978 595 tablet_bootstrap.cc:492] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: Bootstrap starting.
I20260808 10:34:24.788048 597 tablet_bootstrap.cc:492] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: Bootstrap starting.
I20260808 10:34:24.788259 596 tablet_bootstrap.cc:492] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: Bootstrap starting.
I20260808 10:34:24.789480 595 tablet_bootstrap.cc:654] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:24.790282 597 tablet_bootstrap.cc:654] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:24.790282 596 tablet_bootstrap.cc:654] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:24.790362 595 log.cc:824] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:24.791229 597 log.cc:824] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:24.791229 596 log.cc:824] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:24.792452 595 tablet_bootstrap.cc:492] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: No bootstrap required, opened a new log
I20260808 10:34:24.792665 595 ts_tablet_manager.cc:1397] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:34:24.792992 597 tablet_bootstrap.cc:492] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: No bootstrap required, opened a new log
I20260808 10:34:24.793205 597 ts_tablet_manager.cc:1397] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:34:24.795929 595 raft_consensus.cc:348] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } }
I20260808 10:34:24.796206 595 raft_consensus.cc:374] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:24.796269 595 raft_consensus.cc:729] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186d7aca502e493a885b79af4a80dcc8, State: Initialized, Role: FOLLOWER
I20260808 10:34:24.796363 597 raft_consensus.cc:348] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } }
I20260808 10:34:24.796655 597 raft_consensus.cc:374] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:24.796738 597 raft_consensus.cc:729] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54638c91e82f4d1c9243c8137a3b0155, State: Initialized, Role: FOLLOWER
I20260808 10:34:24.796762 595 consensus_queue.cc:260] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } }
I20260808 10:34:24.797266 596 tablet_bootstrap.cc:492] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: No bootstrap required, opened a new log
I20260808 10:34:24.797500 596 ts_tablet_manager.cc:1397] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260808 10:34:24.797231 597 consensus_queue.cc:260] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } }
I20260808 10:34:24.797602 419 heartbeater.cc:503] Master 127.30.146.190:39571 was elected leader, sending a full tablet report...
I20260808 10:34:24.797860 595 ts_tablet_manager.cc:1428] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:34:24.798437 597 ts_tablet_manager.cc:1428] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:34:24.798851 570 heartbeater.cc:503] Master 127.30.146.190:39571 was elected leader, sending a full tablet report...
I20260808 10:34:24.803093 596 raft_consensus.cc:348] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } }
I20260808 10:34:24.803442 596 raft_consensus.cc:374] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:24.803527 596 raft_consensus.cc:729] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a2413a1d86d4b639b1f2d074d6672fc, State: Initialized, Role: FOLLOWER
I20260808 10:34:24.803922 596 consensus_queue.cc:260] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } }
I20260808 10:34:24.804574 32734 heartbeater.cc:503] Master 127.30.146.190:39571 was elected leader, sending a full tablet report...
I20260808 10:34:24.804965 596 ts_tablet_manager.cc:1428] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260808 10:34:24.944504 420 tablet.cc:2369] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:34:24.945005 32735 tablet.cc:2369] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:34:24.959810 571 tablet.cc:2369] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:34:24.997502 601 raft_consensus.cc:482] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:24.997853 601 raft_consensus.cc:504] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } }
I20260808 10:34:25.000612 601 leader_election.cc:288] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54638c91e82f4d1c9243c8137a3b0155 (127.30.146.131:42797), 3a2413a1d86d4b639b1f2d074d6672fc (127.30.146.129:36925)
I20260808 10:34:25.004253 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.999434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:42797 (local address 127.30.146.130:47099)
0808 10:34:24.999749 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:24.999770 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:24.999909 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:25.000314 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:25.000320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:25.000354 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:25.000532 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.000542 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.001741 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.001744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.003033 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.003043 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.003870 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.003873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.003944 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.003965 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:34:25.004059 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 10:34:25.004120 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":31,"thread_start_us":130,"threads_started":1}
I20260808 10:34:25.004300 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:24.999606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:47099 (local address 127.30.146.131:42797)
0808 10:34:24.999752 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:24.999757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:24.999864 (+ 107us) server_negotiation.cc:407] Connection header received
0808 10:34:25.000050 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.000053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.000111 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.000205 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:25.000805 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.001579 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.003184 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.003715 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.003893 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.003947 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.004096 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.004167 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:34:25.004961 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.000149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36925 (local address 127.30.146.130:57313)
0808 10:34:25.000640 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:25.000655 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:25.000818 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:25.001239 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:25.001241 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:25.001259 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:25.001367 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.001375 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.002394 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.002396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.003407 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.003414 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.004595 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.004602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.004698 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.004716 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 10:34:25.004777 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:34:25.004831 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":3,"thread_start_us":74,"threads_started":1}
I20260808 10:34:25.005081 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.000441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:57313 (local address 127.30.146.129:36925)
0808 10:34:25.000599 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.000605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.000723 (+ 118us) server_negotiation.cc:407] Connection header received
0808 10:34:25.000955 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.000958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.001005 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.001103 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:25.001515 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.002241 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.003546 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.004279 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.004463 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.004524 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.004864 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.004968 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:34:25.005396 520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d2fb70d92424afcb57f075925a671f9" candidate_uuid: "186d7aca502e493a885b79af4a80dcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54638c91e82f4d1c9243c8137a3b0155" is_pre_election: true
I20260808 10:34:25.005779 520 raft_consensus.cc:2454] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 186d7aca502e493a885b79af4a80dcc8 in term 0.
I20260808 10:34:25.005743 32685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d2fb70d92424afcb57f075925a671f9" candidate_uuid: "186d7aca502e493a885b79af4a80dcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" is_pre_election: true
I20260808 10:34:25.006021 32685 raft_consensus.cc:2454] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 186d7aca502e493a885b79af4a80dcc8 in term 0.
I20260808 10:34:25.006371 32762 leader_election.cc:302] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 186d7aca502e493a885b79af4a80dcc8, 54638c91e82f4d1c9243c8137a3b0155; no voters:
I20260808 10:34:25.006704 601 raft_consensus.cc:2781] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:25.006788 601 raft_consensus.cc:482] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:25.006877 601 raft_consensus.cc:3037] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:25.008020 601 raft_consensus.cc:504] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } }
I20260808 10:34:25.008363 601 leader_election.cc:288] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [CANDIDATE]: Term 1 election: Requested vote from peers 54638c91e82f4d1c9243c8137a3b0155 (127.30.146.131:42797), 3a2413a1d86d4b639b1f2d074d6672fc (127.30.146.129:36925)
I20260808 10:34:25.008699 520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d2fb70d92424afcb57f075925a671f9" candidate_uuid: "186d7aca502e493a885b79af4a80dcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54638c91e82f4d1c9243c8137a3b0155"
I20260808 10:34:25.008724 32685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d2fb70d92424afcb57f075925a671f9" candidate_uuid: "186d7aca502e493a885b79af4a80dcc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a2413a1d86d4b639b1f2d074d6672fc"
I20260808 10:34:25.008862 32685 raft_consensus.cc:3037] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:25.008859 520 raft_consensus.cc:3037] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:25.010563 32685 raft_consensus.cc:2454] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 186d7aca502e493a885b79af4a80dcc8 in term 1.
I20260808 10:34:25.010950 32763 leader_election.cc:302] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 186d7aca502e493a885b79af4a80dcc8, 3a2413a1d86d4b639b1f2d074d6672fc; no voters:
I20260808 10:34:25.011159 520 raft_consensus.cc:2454] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 186d7aca502e493a885b79af4a80dcc8 in term 1.
I20260808 10:34:25.011416 601 raft_consensus.cc:2781] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:25.011641 601 raft_consensus.cc:686] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 1 LEADER]: Becoming Leader. State: Replica: 186d7aca502e493a885b79af4a80dcc8, State: Running, Role: LEADER
I20260808 10:34:25.012007 601 consensus_queue.cc:237] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } }
I20260808 10:34:25.015683 32514 catalog_manager.cc:5495] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 186d7aca502e493a885b79af4a80dcc8 (127.30.146.130). New cstate: current_term: 1 leader_uuid: "186d7aca502e493a885b79af4a80dcc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:25.077510 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.072387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42144 (local address 127.30.146.130:36369)
0808 10:34:25.072619 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.072625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.072824 (+ 199us) server_negotiation.cc:407] Connection header received
0808 10:34:25.073092 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.073095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.073250 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.073372 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:25.073673 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.074221 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.075265 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.075602 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.075728 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.076230 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:25.076994 (+ 764us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:25.077107 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.077226 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.077289 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 10:34:25.084956 32685 raft_consensus.cc:1261] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 FOLLOWER]: Refusing update from remote peer 186d7aca502e493a885b79af4a80dcc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:25.084956 520 raft_consensus.cc:1261] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 FOLLOWER]: Refusing update from remote peer 186d7aca502e493a885b79af4a80dcc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:25.085929 601 consensus_queue.cc:1035] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:25.088234 601 consensus_queue.cc:1035] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:25.095070 610 mvcc.cc:205] Tried to move back new op lower bound from 7316214845777297408 to 7316214845492236288. Current Snapshot: MvccSnapshot[applied={T|T < 7316214845777297408}]
I20260808 10:34:25.109359 32514 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:25.109649 32514 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:25.112466 32514 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120: Sending DeleteTablet for 3 replicas of tablet 8d2fb70d92424afcb57f075925a671f9
I20260808 10:34:25.113530 32663 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d2fb70d92424afcb57f075925a671f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:25 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:34026
I20260808 10:34:25.113598 500 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d2fb70d92424afcb57f075925a671f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:25 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:52948
I20260808 10:34:25.113538 346 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d2fb70d92424afcb57f075925a671f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:25 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:42136
I20260808 10:34:25.114351 622 tablet_replica.cc:330] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: stopping tablet replica
I20260808 10:34:25.114701 622 raft_consensus.cc:2229] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:34:25.115123 621 tablet_replica.cc:330] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: stopping tablet replica
I20260808 10:34:25.115181 622 raft_consensus.cc:2258] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:25.115120 32514 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:34:25.115464 621 raft_consensus.cc:2229] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 10:34:25.115586 32514 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:25.115772 621 raft_consensus.cc:2258] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:25.116711 622 ts_tablet_manager.cc:1905] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:25.117120 621 ts_tablet_manager.cc:1905] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:25.119822 622 ts_tablet_manager.cc:1918] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:25.119874 621 ts_tablet_manager.cc:1918] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:25.119928 622 log.cc:1196] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/8d2fb70d92424afcb57f075925a671f9
I20260808 10:34:25.119988 621 log.cc:1196] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/8d2fb70d92424afcb57f075925a671f9
I20260808 10:34:25.120415 621 ts_tablet_manager.cc:1939] T 8d2fb70d92424afcb57f075925a671f9 P 3a2413a1d86d4b639b1f2d074d6672fc: Deleting consensus metadata
I20260808 10:34:25.120417 622 ts_tablet_manager.cc:1939] T 8d2fb70d92424afcb57f075925a671f9 P 186d7aca502e493a885b79af4a80dcc8: Deleting consensus metadata
I20260808 10:34:25.122048 32501 catalog_manager.cc:4841] TS 3a2413a1d86d4b639b1f2d074d6672fc (127.30.146.129:36925): tablet 8d2fb70d92424afcb57f075925a671f9 (table test-table [id=07b4e0b575a74ee3844e280fa01d991b]) successfully deleted
I20260808 10:34:25.122126 620 tablet_replica.cc:330] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: stopping tablet replica
I20260808 10:34:25.122356 32663 tablet_service.cc:1467] Processing CreateTablet for tablet 4148cee485e84f48b88bbd8239500f41 (DEFAULT_TABLE table=test-table [id=92cd3642a12e405d84872b3ec506c08b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:25.122475 32500 catalog_manager.cc:4841] TS 186d7aca502e493a885b79af4a80dcc8 (127.30.146.130:36369): tablet 8d2fb70d92424afcb57f075925a671f9 (table test-table [id=07b4e0b575a74ee3844e280fa01d991b]) successfully deleted
I20260808 10:34:25.122661 32663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4148cee485e84f48b88bbd8239500f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:25.122681 346 tablet_service.cc:1467] Processing CreateTablet for tablet 4148cee485e84f48b88bbd8239500f41 (DEFAULT_TABLE table=test-table [id=92cd3642a12e405d84872b3ec506c08b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:25.122926 346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4148cee485e84f48b88bbd8239500f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:25.123046 500 tablet_service.cc:1467] Processing CreateTablet for tablet 4148cee485e84f48b88bbd8239500f41 (DEFAULT_TABLE table=test-table [id=92cd3642a12e405d84872b3ec506c08b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:25.123332 500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4148cee485e84f48b88bbd8239500f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:25.125093 596 tablet_bootstrap.cc:492] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc: Bootstrap starting.
I20260808 10:34:25.125206 595 tablet_bootstrap.cc:492] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8: Bootstrap starting.
I20260808 10:34:25.126219 596 tablet_bootstrap.cc:654] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:25.126232 595 tablet_bootstrap.cc:654] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:25.127069 597 tablet_bootstrap.cc:492] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155: Bootstrap starting.
I20260808 10:34:25.127871 595 tablet_bootstrap.cc:492] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8: No bootstrap required, opened a new log
I20260808 10:34:25.127969 595 ts_tablet_manager.cc:1397] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:25.128345 620 raft_consensus.cc:2229] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:25.128868 597 tablet_bootstrap.cc:654] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:25.128921 620 raft_consensus.cc:2258] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:25.128919 595 raft_consensus.cc:348] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.129041 595 raft_consensus.cc:374] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:25.129097 595 raft_consensus.cc:729] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186d7aca502e493a885b79af4a80dcc8, State: Initialized, Role: FOLLOWER
I20260808 10:34:25.129575 595 consensus_queue.cc:260] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.130559 597 tablet_bootstrap.cc:492] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155: No bootstrap required, opened a new log
I20260808 10:34:25.130673 597 ts_tablet_manager.cc:1397] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260808 10:34:25.130882 596 tablet_bootstrap.cc:492] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc: No bootstrap required, opened a new log
I20260808 10:34:25.130960 596 ts_tablet_manager.cc:1397] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 10:34:25.131254 597 raft_consensus.cc:348] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.131389 597 raft_consensus.cc:374] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:25.131424 597 raft_consensus.cc:729] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54638c91e82f4d1c9243c8137a3b0155, State: Initialized, Role: FOLLOWER
I20260808 10:34:25.131582 597 consensus_queue.cc:260] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.131800 596 raft_consensus.cc:348] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.131913 596 raft_consensus.cc:374] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:25.131938 597 ts_tablet_manager.cc:1428] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260808 10:34:25.131970 596 raft_consensus.cc:729] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a2413a1d86d4b639b1f2d074d6672fc, State: Initialized, Role: FOLLOWER
I20260808 10:34:25.132081 596 consensus_queue.cc:260] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.132318 596 ts_tablet_manager.cc:1428] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:34:25.132432 595 ts_tablet_manager.cc:1428] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260808 10:34:25.134817 620 ts_tablet_manager.cc:1905] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:25.137570 620 ts_tablet_manager.cc:1918] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:25.137703 620 log.cc:1196] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/8d2fb70d92424afcb57f075925a671f9
I20260808 10:34:25.138106 620 ts_tablet_manager.cc:1939] T 8d2fb70d92424afcb57f075925a671f9 P 54638c91e82f4d1c9243c8137a3b0155: Deleting consensus metadata
I20260808 10:34:25.139441 32500 catalog_manager.cc:4841] TS 54638c91e82f4d1c9243c8137a3b0155 (127.30.146.131:42797): tablet 8d2fb70d92424afcb57f075925a671f9 (table test-table [id=07b4e0b575a74ee3844e280fa01d991b]) successfully deleted
I20260808 10:34:25.341293 602 raft_consensus.cc:482] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:25.341468 602 raft_consensus.cc:504] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.342597 602 leader_election.cc:288] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a2413a1d86d4b639b1f2d074d6672fc (127.30.146.129:36925), 186d7aca502e493a885b79af4a80dcc8 (127.30.146.130:36369)
I20260808 10:34:25.347285 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.343139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:43037 (local address 127.30.146.130:36369)
0808 10:34:25.343303 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.343308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.343555 (+ 247us) server_negotiation.cc:407] Connection header received
0808 10:34:25.343739 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.343743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.343815 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.343911 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:25.344272 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.344804 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.346104 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.346564 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.346704 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.346791 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.347050 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.347140 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 10:34:25.347301 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.343238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:49473 (local address 127.30.146.129:36925)
0808 10:34:25.343370 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.343375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.343608 (+ 233us) server_negotiation.cc:407] Connection header received
0808 10:34:25.343768 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.343770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.343813 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.343873 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:25.344203 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.344776 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.346121 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.346565 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.346705 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.346783 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.347106 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.347182 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:34:25.347314 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.342984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:36369 (local address 127.30.146.131:43037)
0808 10:34:25.343431 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:25.343443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:25.343603 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:25.343979 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:25.343982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:25.344000 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:25.344119 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.344124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.344926 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.344932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.345973 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.345983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.346696 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.346700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.346889 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.346920 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 10:34:25.347033 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 10:34:25.347142 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":74,"thread_start_us":131,"threads_started":1}
I20260808 10:34:25.347316 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.342984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36925 (local address 127.30.146.131:49473)
0808 10:34:25.343464 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:25.343476 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:25.343623 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:25.343916 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:25.343919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:25.343936 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:25.344085 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.344091 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.344899 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.344903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.345972 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.345982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.346697 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.346700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.346937 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.346965 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 10:34:25.347071 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 10:34:25.347159 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":291,"thread_start_us":83,"threads_started":1}
I20260808 10:34:25.348255 32685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4148cee485e84f48b88bbd8239500f41" candidate_uuid: "54638c91e82f4d1c9243c8137a3b0155" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" is_pre_election: true
I20260808 10:34:25.348242 367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4148cee485e84f48b88bbd8239500f41" candidate_uuid: "54638c91e82f4d1c9243c8137a3b0155" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186d7aca502e493a885b79af4a80dcc8" is_pre_election: true
I20260808 10:34:25.348414 32685 raft_consensus.cc:2454] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54638c91e82f4d1c9243c8137a3b0155 in term 0.
I20260808 10:34:25.348543 367 raft_consensus.cc:2454] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54638c91e82f4d1c9243c8137a3b0155 in term 0.
I20260808 10:34:25.348922 448 leader_election.cc:302] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a2413a1d86d4b639b1f2d074d6672fc, 54638c91e82f4d1c9243c8137a3b0155; no voters:
I20260808 10:34:25.349216 602 raft_consensus.cc:2781] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:25.349279 602 raft_consensus.cc:482] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:25.349318 602 raft_consensus.cc:3037] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:25.350515 602 raft_consensus.cc:504] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.350901 602 leader_election.cc:288] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [CANDIDATE]: Term 1 election: Requested vote from peers 3a2413a1d86d4b639b1f2d074d6672fc (127.30.146.129:36925), 186d7aca502e493a885b79af4a80dcc8 (127.30.146.130:36369)
I20260808 10:34:25.351364 32685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4148cee485e84f48b88bbd8239500f41" candidate_uuid: "54638c91e82f4d1c9243c8137a3b0155" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a2413a1d86d4b639b1f2d074d6672fc"
I20260808 10:34:25.351507 32685 raft_consensus.cc:3037] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:25.351531 367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4148cee485e84f48b88bbd8239500f41" candidate_uuid: "54638c91e82f4d1c9243c8137a3b0155" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186d7aca502e493a885b79af4a80dcc8"
I20260808 10:34:25.351850 367 raft_consensus.cc:3037] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:25.352628 32685 raft_consensus.cc:2454] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54638c91e82f4d1c9243c8137a3b0155 in term 1.
I20260808 10:34:25.352917 448 leader_election.cc:302] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a2413a1d86d4b639b1f2d074d6672fc, 54638c91e82f4d1c9243c8137a3b0155; no voters:
I20260808 10:34:25.353122 602 raft_consensus.cc:2781] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:25.353318 367 raft_consensus.cc:2454] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54638c91e82f4d1c9243c8137a3b0155 in term 1.
I20260808 10:34:25.353359 602 raft_consensus.cc:686] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 LEADER]: Becoming Leader. State: Replica: 54638c91e82f4d1c9243c8137a3b0155, State: Running, Role: LEADER
I20260808 10:34:25.353690 602 consensus_queue.cc:237] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.356317 32513 catalog_manager.cc:5495] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54638c91e82f4d1c9243c8137a3b0155 (127.30.146.131). New cstate: current_term: 1 leader_uuid: "54638c91e82f4d1c9243c8137a3b0155" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } health_report { overall_health: HEALTHY } } }
Aug 08 10:34:25 dist-test-slave-9gcw krb5kdc[32467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:25.409087 637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.399340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39571 (local address 127.0.0.1:59714)
0808 10:34:25.399691 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:25.399711 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:25.399846 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:25.400176 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:25.400179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:25.400655 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:25.400753 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.400761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.401852 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.401857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.402423 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.402433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.402978 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.402981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.403064 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.403641 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:25.403654 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:25.405272 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:25.407158 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:25.407168 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:25.407175 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:25.407438 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:25.407795 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:25.407798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:25.407800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:25.407893 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:25.408319 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:25.408327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:25.408448 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 10:34:25.408598 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 10:34:25.408765 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":96,"threads_started":1}
I20260808 10:34:25.409087 638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.399368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59714 (local address 127.30.146.190:39571)
0808 10:34:25.399764 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.399790 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.399806 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:34:25.399990 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.399992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.400033 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.400162 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:25.400908 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.401719 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.402556 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.402803 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.402958 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.405413 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:25.405416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:25.405436 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:25.405460 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:25.407026 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:25.407546 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:25.407564 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:25.407566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:25.407689 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:25.407987 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:25.407993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:25.407994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:25.408175 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:25.408328 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.408652 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.408860 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":325,"thread_start_us":199,"threads_started":1}
I20260808 10:34:25.413944 32513 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:59714:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:34:25.414824 32513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:25.419696 346 tablet_service.cc:1467] Processing CreateTablet for tablet 8425da9c952b49e6b020ccd38a544f10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ca8cf45f94f488287f34af3ecc30447]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:34:25.419685 32663 tablet_service.cc:1467] Processing CreateTablet for tablet 8425da9c952b49e6b020ccd38a544f10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ca8cf45f94f488287f34af3ecc30447]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:34:25.419838 500 tablet_service.cc:1467] Processing CreateTablet for tablet 8425da9c952b49e6b020ccd38a544f10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ca8cf45f94f488287f34af3ecc30447]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:34:25.420171 32663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8425da9c952b49e6b020ccd38a544f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:25.420173 346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8425da9c952b49e6b020ccd38a544f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:25.420212 500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8425da9c952b49e6b020ccd38a544f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:25.423163 595 tablet_bootstrap.cc:492] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: Bootstrap starting.
I20260808 10:34:25.423602 597 tablet_bootstrap.cc:492] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: Bootstrap starting.
I20260808 10:34:25.424209 595 tablet_bootstrap.cc:654] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:25.425128 597 tablet_bootstrap.cc:654] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:25.425535 595 tablet_bootstrap.cc:492] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: No bootstrap required, opened a new log
I20260808 10:34:25.425632 595 ts_tablet_manager.cc:1397] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:25.426087 595 raft_consensus.cc:348] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.426296 595 raft_consensus.cc:374] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:25.426354 595 raft_consensus.cc:729] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186d7aca502e493a885b79af4a80dcc8, State: Initialized, Role: FOLLOWER
I20260808 10:34:25.426488 595 consensus_queue.cc:260] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.426685 596 tablet_bootstrap.cc:492] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: Bootstrap starting.
I20260808 10:34:25.426730 601 tablet_replica.cc:439] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } }
I20260808 10:34:25.426877 601 tablet_replica.cc:442] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:25.427448 597 tablet_bootstrap.cc:492] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: No bootstrap required, opened a new log
I20260808 10:34:25.427534 595 ts_tablet_manager.cc:1428] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:34:25.427574 597 ts_tablet_manager.cc:1397] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260808 10:34:25.428355 596 tablet_bootstrap.cc:654] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:25.428390 597 raft_consensus.cc:348] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.428498 597 raft_consensus.cc:374] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:25.428544 597 raft_consensus.cc:729] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54638c91e82f4d1c9243c8137a3b0155, State: Initialized, Role: FOLLOWER
I20260808 10:34:25.428774 597 consensus_queue.cc:260] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.429070 597 ts_tablet_manager.cc:1428] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:34:25.429656 602 tablet_replica.cc:439] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } }
I20260808 10:34:25.429802 602 tablet_replica.cc:442] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:25.431306 596 tablet_bootstrap.cc:492] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: No bootstrap required, opened a new log
I20260808 10:34:25.431396 596 ts_tablet_manager.cc:1397] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 10:34:25.432094 596 raft_consensus.cc:348] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.432220 596 raft_consensus.cc:374] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:25.432269 596 raft_consensus.cc:729] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a2413a1d86d4b639b1f2d074d6672fc, State: Initialized, Role: FOLLOWER
I20260808 10:34:25.432390 596 consensus_queue.cc:260] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.432677 603 tablet_replica.cc:439] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } }
I20260808 10:34:25.432883 603 tablet_replica.cc:442] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:25.432674 596 ts_tablet_manager.cc:1428] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:34:25.503039 603 raft_consensus.cc:482] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:25.503194 603 raft_consensus.cc:504] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.504330 603 leader_election.cc:288] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 186d7aca502e493a885b79af4a80dcc8 (127.30.146.130:36369), 54638c91e82f4d1c9243c8137a3b0155 (127.30.146.131:42797)
I20260808 10:34:25.509665 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.504733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:54179 (local address 127.30.146.130:36369)
0808 10:34:25.504916 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.504921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.505269 (+ 348us) server_negotiation.cc:407] Connection header received
0808 10:34:25.505536 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.505541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.505605 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.505722 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:25.506188 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.506827 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.508488 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.508910 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.509055 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.509138 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.509449 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.509544 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:34:25.509742 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.504616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:36369 (local address 127.30.146.129:54179)
0808 10:34:25.505121 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:25.505140 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:25.505367 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:25.505735 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:25.505738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:25.505756 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:25.506048 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.506055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.507001 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.507004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.508348 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.508358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.509049 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.509053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.509297 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.509326 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 10:34:25.509430 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 10:34:25.509520 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"spinlock_wait_cycles":32512,"thread_start_us":254,"threads_started":1}
I20260808 10:34:25.509769 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.505359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:56355 (local address 127.30.146.131:42797)
0808 10:34:25.505671 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.505676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.505812 (+ 136us) server_negotiation.cc:407] Connection header received
0808 10:34:25.505976 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.505979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.506023 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.506135 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:25.506582 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.507214 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.508585 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.509007 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.509143 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.509230 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.509578 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.509644 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":113,"threads_started":1}
I20260808 10:34:25.509810 644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.505208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:42797 (local address 127.30.146.129:56355)
0808 10:34:25.505706 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:25.505719 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:25.505883 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:25.506144 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:25.506147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:25.506163 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:25.506463 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.506471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.507455 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.507458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.508441 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.508452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.509138 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.509165 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.509390 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.509410 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:34:25.509538 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 10:34:25.509614 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":221,"threads_started":1}
I20260808 10:34:25.510171 367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8425da9c952b49e6b020ccd38a544f10" candidate_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186d7aca502e493a885b79af4a80dcc8" is_pre_election: true
I20260808 10:34:25.510311 367 raft_consensus.cc:2454] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a2413a1d86d4b639b1f2d074d6672fc in term 0.
I20260808 10:34:25.510393 520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8425da9c952b49e6b020ccd38a544f10" candidate_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54638c91e82f4d1c9243c8137a3b0155" is_pre_election: true
I20260808 10:34:25.510543 520 raft_consensus.cc:2454] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a2413a1d86d4b639b1f2d074d6672fc in term 0.
I20260808 10:34:25.510771 32612 leader_election.cc:302] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 186d7aca502e493a885b79af4a80dcc8, 3a2413a1d86d4b639b1f2d074d6672fc; no voters:
I20260808 10:34:25.511073 603 raft_consensus.cc:2781] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:25.511173 603 raft_consensus.cc:482] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:25.511265 603 raft_consensus.cc:3037] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:25.512653 603 raft_consensus.cc:504] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.513032 603 leader_election.cc:288] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [CANDIDATE]: Term 1 election: Requested vote from peers 186d7aca502e493a885b79af4a80dcc8 (127.30.146.130:36369), 54638c91e82f4d1c9243c8137a3b0155 (127.30.146.131:42797)
I20260808 10:34:25.513411 367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8425da9c952b49e6b020ccd38a544f10" candidate_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186d7aca502e493a885b79af4a80dcc8"
I20260808 10:34:25.513563 367 raft_consensus.cc:3037] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:25.513543 520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8425da9c952b49e6b020ccd38a544f10" candidate_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54638c91e82f4d1c9243c8137a3b0155"
I20260808 10:34:25.513669 520 raft_consensus.cc:3037] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:25.514648 367 raft_consensus.cc:2454] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a2413a1d86d4b639b1f2d074d6672fc in term 1.
I20260808 10:34:25.514979 32612 leader_election.cc:302] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 186d7aca502e493a885b79af4a80dcc8, 3a2413a1d86d4b639b1f2d074d6672fc; no voters:
I20260808 10:34:25.515184 603 raft_consensus.cc:2781] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:25.515208 520 raft_consensus.cc:2454] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a2413a1d86d4b639b1f2d074d6672fc in term 1.
I20260808 10:34:25.515458 603 raft_consensus.cc:686] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 LEADER]: Becoming Leader. State: Replica: 3a2413a1d86d4b639b1f2d074d6672fc, State: Running, Role: LEADER
I20260808 10:34:25.515760 603 consensus_queue.cc:237] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } }
I20260808 10:34:25.517138 646 tablet_replica.cc:439] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: TxnStatusTablet state changed. Reason: New leader 3a2413a1d86d4b639b1f2d074d6672fc. Latest consensus state: current_term: 1 leader_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } }
I20260808 10:34:25.517304 646 tablet_replica.cc:442] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:34:25.518054 648 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:34:25.518252 648 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:34:25.518846 32515 catalog_manager.cc:5495] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a2413a1d86d4b639b1f2d074d6672fc (127.30.146.129). New cstate: current_term: 1 leader_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:25.557400 637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.552221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36925 (local address 127.0.0.1:34034)
0808 10:34:25.552425 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:25.552448 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:25.552545 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:25.552970 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:25.552973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:25.552980 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:25.553079 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.553084 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.554356 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.554359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.554984 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.554994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.555857 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.555860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.555930 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.555976 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:25.557028 (+ 1052us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:25.557043 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 10:34:25.557092 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 10:34:25.557246 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 10:34:25.557533 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.552354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34034 (local address 127.30.146.129:36925)
0808 10:34:25.552508 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.552514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.552535 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:34:25.552716 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.552720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.552803 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.552932 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:25.553269 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.554201 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.555171 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.555722 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.555860 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.556149 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:25.556911 (+ 762us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:25.557064 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.557326 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.557396 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:34:25.561826 520 raft_consensus.cc:1261] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 FOLLOWER]: Refusing update from remote peer 3a2413a1d86d4b639b1f2d074d6672fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:25.562045 367 raft_consensus.cc:1261] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8 [term 1 FOLLOWER]: Refusing update from remote peer 3a2413a1d86d4b639b1f2d074d6672fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:25.562593 646 consensus_queue.cc:1035] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:25.562911 603 consensus_queue.cc:1035] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:25.564963 602 tablet_replica.cc:439] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: TxnStatusTablet state changed. Reason: New leader 3a2413a1d86d4b639b1f2d074d6672fc. Latest consensus state: current_term: 1 leader_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } }
I20260808 10:34:25.565116 602 tablet_replica.cc:442] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:25.566310 601 tablet_replica.cc:439] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: TxnStatusTablet state changed. Reason: New leader 3a2413a1d86d4b639b1f2d074d6672fc. Latest consensus state: current_term: 1 leader_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } }
I20260808 10:34:25.566432 601 tablet_replica.cc:442] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:25.569450 646 tablet_replica.cc:439] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } }
I20260808 10:34:25.569614 646 tablet_replica.cc:442] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:34:25.569839 646 tablet_replica.cc:439] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } }
I20260808 10:34:25.569948 646 tablet_replica.cc:442] T 8425da9c952b49e6b020ccd38a544f10 P 3a2413a1d86d4b639b1f2d074d6672fc: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:34:25.570067 611 mvcc.cc:205] Tried to move back new op lower bound from 7316214847735386112 to 7316214847554891776. Current Snapshot: MvccSnapshot[applied={T|T < 7316214847735386112 or (T in {7316214847735386112})}]
I20260808 10:34:25.570225 601 tablet_replica.cc:439] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } }
I20260808 10:34:25.570354 601 tablet_replica.cc:442] T 8425da9c952b49e6b020ccd38a544f10 P 186d7aca502e493a885b79af4a80dcc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:25.572726 613 mvcc.cc:205] Tried to move back new op lower bound from 7316214847735386112 to 7316214847554891776. Current Snapshot: MvccSnapshot[applied={T|T < 7316214847735386112}]
I20260808 10:34:25.574733 602 tablet_replica.cc:439] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 } } peers { permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 } } peers { permanent_uuid: "54638c91e82f4d1c9243c8137a3b0155" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42797 } } }
I20260808 10:34:25.575019 602 tablet_replica.cc:442] T 8425da9c952b49e6b020ccd38a544f10 P 54638c91e82f4d1c9243c8137a3b0155: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:25.585718 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.580747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.30.146.131:42797)
0808 10:34:25.580905 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.580910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.581249 (+ 339us) server_negotiation.cc:407] Connection header received
0808 10:34:25.581558 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.581561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.581614 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.581701 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:25.582041 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.582595 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.583994 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.584194 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.584269 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.584680 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:25.585242 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:25.585350 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.585463 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.585510 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:34:25.602662 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.596248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:34335 (local address 127.30.146.129:36925)
0808 10:34:25.596462 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.596468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.596675 (+ 207us) server_negotiation.cc:407] Connection header received
0808 10:34:25.596820 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.596824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.596884 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.596976 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:25.597611 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.598626 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.601286 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.601498 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.601584 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.602068 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:25.602155 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:25.602272 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.602449 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.602498 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 10:34:25.602734 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.595934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36925 (local address 127.30.146.131:34335)
0808 10:34:25.596565 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:25.596580 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:25.596693 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:25.596993 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:25.596995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:25.597009 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:25.597446 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.597455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.598906 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.598910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.601097 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.601114 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.601647 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.601652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.601828 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.601893 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:25.602311 (+ 418us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:25.602320 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:34:25.602404 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:34:25.602488 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":234,"threads_started":1}
I20260808 10:34:25.612159 32685 raft_consensus.cc:1261] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 FOLLOWER]: Refusing update from remote peer 54638c91e82f4d1c9243c8137a3b0155: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:25.612265 367 raft_consensus.cc:1261] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8 [term 1 FOLLOWER]: Refusing update from remote peer 54638c91e82f4d1c9243c8137a3b0155: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:25.612808 602 consensus_queue.cc:1035] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [LEADER]: Connected to new peer: Peer: permanent_uuid: "186d7aca502e493a885b79af4a80dcc8" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:25.613284 602 consensus_queue.cc:1035] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a2413a1d86d4b639b1f2d074d6672fc" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:25.639523 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.635132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:45085 (local address 127.30.146.131:42797)
0808 10:34:25.635282 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.635286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.635426 (+ 140us) server_negotiation.cc:407] Connection header received
0808 10:34:25.635614 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.635617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.635666 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.635732 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:25.636206 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.636722 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.638136 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.638471 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.638551 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.639015 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:25.639096 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:25.639186 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.639284 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.639336 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:34:25.639904 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.634646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:42797 (local address 127.30.146.129:45085)
0808 10:34:25.635338 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:25.635359 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:25.635491 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:25.635780 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:25.635784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:25.635793 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:25.636063 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.636083 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.636853 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.636858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.637998 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:25.638011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.638661 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.638663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:25.638798 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.638860 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:25.639194 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:25.639203 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:34:25.639268 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:34:25.639353 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":191,"threads_started":1}
I20260808 10:34:25.670533 32513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:25.670746 32513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:25.673763 32513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84a8357ae09f4262a6d6250c4020b120: Sending DeleteTablet for 3 replicas of tablet 4148cee485e84f48b88bbd8239500f41
I20260808 10:34:25.674997 32663 tablet_service.cc:1514] Processing DeleteTablet for tablet 4148cee485e84f48b88bbd8239500f41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:25 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:34026
I20260808 10:34:25.675452 346 tablet_service.cc:1514] Processing DeleteTablet for tablet 4148cee485e84f48b88bbd8239500f41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:25 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:42136
I20260808 10:34:25.676630 500 tablet_service.cc:1514] Processing DeleteTablet for tablet 4148cee485e84f48b88bbd8239500f41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:25 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:52948
I20260808 10:34:25.677407 670 tablet_replica.cc:330] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8: stopping tablet replica
I20260808 10:34:25.677721 672 tablet_replica.cc:330] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155: stopping tablet replica
I20260808 10:34:25.678004 670 raft_consensus.cc:2229] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:25.678260 668 tablet_replica.cc:330] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc: stopping tablet replica
I20260808 10:34:25.678472 672 raft_consensus.cc:2229] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:34:25.678695 670 raft_consensus.cc:2258] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:25.678900 668 raft_consensus.cc:2229] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:25.679352 672 raft_consensus.cc:2258] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:25.679711 668 raft_consensus.cc:2258] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:25.681352 668 ts_tablet_manager.cc:1905] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:25.682209 672 ts_tablet_manager.cc:1905] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:25.682631 670 ts_tablet_manager.cc:1905] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 10:34:25 dist-test-slave-9gcw krb5kdc[32467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185264, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:34:25.691969 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.676158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34046 (local address 127.30.146.129:36925)
0808 10:34:25.676334 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.676339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.677133 (+ 794us) server_negotiation.cc:407] Connection header received
0808 10:34:25.677732 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.677736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.677785 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.677838 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:25.679069 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.679920 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.681799 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.682160 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.682239 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.687053 (+ 4814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:25.687064 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:25.687078 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:25.687105 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:25.689517 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:25.690071 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:25.690077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:25.690079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:25.690247 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:25.690564 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:25.690567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:25.690569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:25.690892 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:25.691079 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.691527 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.691656 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 10:34:25.692981 32728 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34046
I20260808 10:34:25.693931 668 ts_tablet_manager.cc:1918] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:34:25.694043 668 log.cc:1196] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/4148cee485e84f48b88bbd8239500f41
I20260808 10:34:25.694448 668 ts_tablet_manager.cc:1939] T 4148cee485e84f48b88bbd8239500f41 P 3a2413a1d86d4b639b1f2d074d6672fc: Deleting consensus metadata
I20260808 10:34:25.695130 670 ts_tablet_manager.cc:1918] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:34:25.695223 670 log.cc:1196] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/4148cee485e84f48b88bbd8239500f41
I20260808 10:34:25.695580 670 ts_tablet_manager.cc:1939] T 4148cee485e84f48b88bbd8239500f41 P 186d7aca502e493a885b79af4a80dcc8: Deleting consensus metadata
I20260808 10:34:25.695775 672 ts_tablet_manager.cc:1918] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:34:25.695946 672 log.cc:1196] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/4148cee485e84f48b88bbd8239500f41
I20260808 10:34:25.697081 32501 catalog_manager.cc:4841] TS 3a2413a1d86d4b639b1f2d074d6672fc (127.30.146.129:36925): tablet 4148cee485e84f48b88bbd8239500f41 (table test-table [id=92cd3642a12e405d84872b3ec506c08b]) successfully deleted
I20260808 10:34:25.697420 32500 catalog_manager.cc:4841] TS 186d7aca502e493a885b79af4a80dcc8 (127.30.146.130:36369): tablet 4148cee485e84f48b88bbd8239500f41 (table test-table [id=92cd3642a12e405d84872b3ec506c08b]) successfully deleted
I20260808 10:34:25.697808 672 ts_tablet_manager.cc:1939] T 4148cee485e84f48b88bbd8239500f41 P 54638c91e82f4d1c9243c8137a3b0155: Deleting consensus metadata
I20260808 10:34:25.699443 32500 catalog_manager.cc:4841] TS 54638c91e82f4d1c9243c8137a3b0155 (127.30.146.131:42797): tablet 4148cee485e84f48b88bbd8239500f41 (table test-table [id=92cd3642a12e405d84872b3ec506c08b]) successfully deleted
I20260808 10:34:25.711359 638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:25.697565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.30.146.190:39571)
0808 10:34:25.697774 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:25.697780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:25.698197 (+ 417us) server_negotiation.cc:407] Connection header received
0808 10:34:25.698419 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:25.698429 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:25.698483 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:25.698541 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:25.699586 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.700463 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.701918 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:25.702459 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:25.702629 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:25.704659 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:25.704664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:25.704667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:25.704702 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:25.707639 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:25.708522 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:25.708526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:25.708528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:25.708825 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:25.709377 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:25.709381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:25.709383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:25.709818 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:25.710226 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:25.710857 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 10:34:25.711125 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 10:34:25.712787 32513 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59728
I20260808 10:34:25.715266 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 32593
I20260808 10:34:25.733065 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 32743
I20260808 10:34:25.743381 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 429
I20260808 10:34:25.752049 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 32481
2026-08-08T10:34:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3532 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:25 dist-test-slave-9gcw krb5kdc[677](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:25 dist-test-slave-9gcw krb5kdc[677](info): set up 2 sockets
Aug 08 10:34:25 dist-test-slave-9gcw krb5kdc[677](info): commencing operation
krb5kdc: starting...
W20260808 10:34:26.897079 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:26 dist-test-slave-9gcw krb5kdc[677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:34:27.041743 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:44357
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:44357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:27.159658 691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:27.159988 691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:27.160073 691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:27.164417 691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:27.164535 691 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:34:27.164568 691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:27.164610 691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:27.164644 691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:27.164686 691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:27.170895 691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:44357
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:44357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:27.172367 691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:27.173982 691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:27.185801 696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:27.185904 697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:27.185966 691 server_base.cc:1034] running on GCE node
W20260808 10:34:27.185809 699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:27.186682 691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:27.187933 691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:27.189111 691 hybrid_clock.cc:648] HybridClock initialized: now 1786185267189097 us; error 27 us; skew 500 ppm
Aug 08 10:34:27 dist-test-slave-9gcw krb5kdc[677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:27.195422 691 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:34:27.196502 691 webserver.cc:460] Webserver started at http://127.30.146.190:46625/ using document root <none> and password file <none>
I20260808 10:34:27.197047 691 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:27.197125 691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:27.197433 691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:27.199596 691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "1e9315f5ee484844b5e34f76f3ef0d8f"
format_stamp: "Formatted at 2026-08-08 10:34:27 on dist-test-slave-9gcw"
server_key: "67c89a21d5551fdd0fb10be8c48a20f8"
server_key_iv: "6f6911bc78df0531c3dc6a7a48fe8487"
server_key_version: "encryptionkey@0"
I20260808 10:34:27.200184 691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "1e9315f5ee484844b5e34f76f3ef0d8f"
format_stamp: "Formatted at 2026-08-08 10:34:27 on dist-test-slave-9gcw"
server_key: "67c89a21d5551fdd0fb10be8c48a20f8"
server_key_iv: "6f6911bc78df0531c3dc6a7a48fe8487"
server_key_version: "encryptionkey@0"
I20260808 10:34:27.203994 691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:34:27.206602 707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:27.207887 691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:34:27.208034 691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "1e9315f5ee484844b5e34f76f3ef0d8f"
format_stamp: "Formatted at 2026-08-08 10:34:27 on dist-test-slave-9gcw"
server_key: "67c89a21d5551fdd0fb10be8c48a20f8"
server_key_iv: "6f6911bc78df0531c3dc6a7a48fe8487"
server_key_version: "encryptionkey@0"
I20260808 10:34:27.208154 691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:27.223088 691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:27.226207 691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:27.226454 691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:27.235294 691 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:44357
I20260808 10:34:27.235349 773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:44357 every 8 connection(s)
I20260808 10:34:27.236330 691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:27.239377 774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:27.239866 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 691
I20260808 10:34:27.240013 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:27.240294 31306 external_mini_cluster.cc:1428] Setting key 4de2b00bff7f35f7259b21c2eea00ad2
I20260808 10:34:27.247046 774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f: Bootstrap starting.
I20260808 10:34:27.249670 774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f: Neither blocks nor log segments found. Creating new log.
Aug 08 10:34:27 dist-test-slave-9gcw krb5kdc[677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:27.250555 774 log.cc:824] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:27.252624 774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f: No bootstrap required, opened a new log
I20260808 10:34:27.255937 774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9315f5ee484844b5e34f76f3ef0d8f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44357 } }
I20260808 10:34:27.256187 774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:27.256259 774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e9315f5ee484844b5e34f76f3ef0d8f, State: Initialized, Role: FOLLOWER
I20260808 10:34:27.256688 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:27.241795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.30.146.190:44357)
0808 10:34:27.242246 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:27.242266 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:27.242314 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:34:27.243071 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:27.243095 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:27.243552 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:27.243924 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:27.244669 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.246296 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.247186 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.247849 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.248008 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:27.250783 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:27.250789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:27.250804 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:27.250834 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:27.253520 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:27.254009 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:27.254020 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:27.254024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:27.254149 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:27.254505 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:27.254508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:27.254509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:27.254920 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:27.255164 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:27.255450 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 10:34:27.255768 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":117,"threads_started":1}
I20260808 10:34:27.256701 774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9315f5ee484844b5e34f76f3ef0d8f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44357 } }
I20260808 10:34:27.256939 774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:27.257004 774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:27.257077 774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:27.258165 774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9315f5ee484844b5e34f76f3ef0d8f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44357 } }
I20260808 10:34:27.258522 774 leader_election.cc:302] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e9315f5ee484844b5e34f76f3ef0d8f; no voters:
I20260808 10:34:27.258873 774 leader_election.cc:288] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:27.258952 780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:27.259157 780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [term 1 LEADER]: Becoming Leader. State: Replica: 1e9315f5ee484844b5e34f76f3ef0d8f, State: Running, Role: LEADER
I20260808 10:34:27.259447 780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9315f5ee484844b5e34f76f3ef0d8f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44357 } }
I20260808 10:34:27.260187 774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:27.260741 780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e9315f5ee484844b5e34f76f3ef0d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9315f5ee484844b5e34f76f3ef0d8f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44357 } } }
I20260808 10:34:27.260895 780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:27.261400 787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:27.261685 781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e9315f5ee484844b5e34f76f3ef0d8f. Latest consensus state: current_term: 1 leader_uuid: "1e9315f5ee484844b5e34f76f3ef0d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9315f5ee484844b5e34f76f3ef0d8f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44357 } } }
I20260808 10:34:27.261812 781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:27.265012 787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:27.272147 787 catalog_manager.cc:1347] Generated new cluster ID: 6a29770c4d7a4afeb26966c284abc871
I20260808 10:34:27.272277 787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:27.289867 787 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:27.291386 787 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:27.296478 787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f: Generated new TSK 0
I20260808 10:34:27.297261 787 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:34:27.338121 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:44357
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:27.494660 803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:27.494931 803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:27.494980 803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:27.499295 803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:27.499359 803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:27.499400 803 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:34:27.499445 803 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:34:27.499507 803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:27.504941 803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.146.190:44357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:27.506161 803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:27.507373 803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:34:27.517671 803 server_base.cc:1034] running on GCE node
W20260808 10:34:27.517621 809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:27.517683 811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:27.517621 808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:27.518515 803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:27.519145 803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:27.520308 803 hybrid_clock.cc:648] HybridClock initialized: now 1786185267520294 us; error 63 us; skew 500 ppm
Aug 08 10:34:27 dist-test-slave-9gcw krb5kdc[677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:27.523370 803 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:34:27.524367 803 webserver.cc:460] Webserver started at http://127.30.146.129:43291/ using document root <none> and password file <none>
I20260808 10:34:27.524981 803 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:27.525048 803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:27.525321 803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:27.527504 803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "a3fa61780cab4954aaa4f257de1bf5be"
format_stamp: "Formatted at 2026-08-08 10:34:27 on dist-test-slave-9gcw"
server_key: "cd154570770d0a465ba5e3d226080e7b"
server_key_iv: "fa6c81a7bb93c2d345c4e5eb047fe030"
server_key_version: "encryptionkey@0"
I20260808 10:34:27.528098 803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "a3fa61780cab4954aaa4f257de1bf5be"
format_stamp: "Formatted at 2026-08-08 10:34:27 on dist-test-slave-9gcw"
server_key: "cd154570770d0a465ba5e3d226080e7b"
server_key_iv: "fa6c81a7bb93c2d345c4e5eb047fe030"
server_key_version: "encryptionkey@0"
I20260808 10:34:27.532219 803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:34:27.535028 818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:27.536366 803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:27.536471 803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "a3fa61780cab4954aaa4f257de1bf5be"
format_stamp: "Formatted at 2026-08-08 10:34:27 on dist-test-slave-9gcw"
server_key: "cd154570770d0a465ba5e3d226080e7b"
server_key_iv: "fa6c81a7bb93c2d345c4e5eb047fe030"
server_key_version: "encryptionkey@0"
I20260808 10:34:27.536638 803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:27.555258 803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:27.559026 803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:27.559247 803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:27.561244 803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:27.561326 803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:27.561398 803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:27.561427 803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:34:27 dist-test-slave-9gcw krb5kdc[677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:27.573082 803 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:34031
I20260808 10:34:27.573511 946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:34031 every 8 connection(s)
I20260808 10:34:27.574242 803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:27.576443 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:27.561922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:56279 (local address 127.30.146.190:44357)
0808 10:34:27.562078 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:27.562085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:27.563509 (+ 1424us) server_negotiation.cc:407] Connection header received
0808 10:34:27.564889 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:27.564897 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:27.564969 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:27.565084 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:27.566416 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.567102 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.569039 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.569465 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.569579 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:27.572494 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:27.572498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:27.572501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:27.572530 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:27.574142 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:27.574781 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:27.574785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:27.574786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:27.574895 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:27.575338 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:27.575342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:27.575344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:27.575572 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:27.575657 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:27.576178 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 10:34:27.576308 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:34:27.577360 828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:27.562794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:44357 (local address 127.30.146.129:56279)
0808 10:34:27.563321 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:27.563374 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:27.564553 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:27.565273 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:27.565281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:27.565767 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:27.566253 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:27.566267 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.567258 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.567262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:27.568873 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:27.568888 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.569617 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.569621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:27.569803 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:27.570455 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:27.570471 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:27.572276 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:27.574263 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:27.574270 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:27.574285 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:27.574652 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:27.575012 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:27.575017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:27.575018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:27.575192 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:27.575678 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:27.575682 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:27.575919 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 10:34:27.576078 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 10:34:27.576368 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":194,"threads_started":1}
I20260808 10:34:27.577827 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 803
I20260808 10:34:27.577993 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:27.578295 31306 external_mini_cluster.cc:1428] Setting key e73f6f5a5d27206c718fc9f80c222451
I20260808 10:34:27.578877 947 heartbeater.cc:347] Connected to a master server at 127.30.146.190:44357
I20260808 10:34:27.579110 947 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:27.579696 947 heartbeater.cc:511] Master 127.30.146.190:44357 requested a full tablet report, sending...
I20260808 10:34:27.581365 725 ts_manager.cc:194] Registered new tserver with Master: a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031)
I20260808 10:34:27.582703 725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:56279
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:34:27.590901 953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:27.581581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:44357 (local address 127.30.146.129:42983)
0808 10:34:27.581889 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:27.581905 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:27.581992 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:27.582658 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:27.582662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:27.582951 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:27.583132 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:27.583140 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.584274 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.584280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:27.585395 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:27.585417 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.585898 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.585902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:27.586026 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:27.586628 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:27.586661 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:27.587066 (+ 405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:27.589227 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:27.589231 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:27.589237 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:27.589438 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:27.589767 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:27.589769 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:27.589771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:27.589870 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:27.590347 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:27.590349 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:27.590427 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 10:34:27.590507 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:34:27.590671 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":101,"threads_started":1}
I20260808 10:34:27.591007 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:27.581930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:42983 (local address 127.30.146.190:44357)
0808 10:34:27.582397 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:27.582402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:27.582419 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:34:27.582459 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:27.582464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:27.582516 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:27.582620 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:27.583286 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.584099 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.585546 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.585757 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.585875 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:27.587237 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:27.587243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:27.587249 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:27.587283 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:27.589019 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:27.589552 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:27.589555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:27.589557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:27.589656 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:27.589976 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:27.589982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:27.589987 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:27.590234 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:27.590353 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:27.590615 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 10:34:27.590750 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:34:27.613013 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:44357
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:27.730454 957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:27.730710 957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:27.730803 957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:27.734941 957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:27.735001 957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:27.735071 957 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:34:27.735105 957 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:34:27.735162 957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:27.740545 957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.146.190:44357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:27.741678 957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:27.742765 957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:27.752839 965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:27.752862 962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:27.752851 963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:27.753043 957 server_base.cc:1034] running on GCE node
I20260808 10:34:27.753718 957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:27.754419 957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:27.755589 957 hybrid_clock.cc:648] HybridClock initialized: now 1786185267755552 us; error 50 us; skew 500 ppm
Aug 08 10:34:27 dist-test-slave-9gcw krb5kdc[677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:27.758944 957 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:34:27.759876 957 webserver.cc:460] Webserver started at http://127.30.146.130:46583/ using document root <none> and password file <none>
I20260808 10:34:27.760581 957 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:27.760681 957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:27.760932 957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:27.763438 957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "4c120b5093954ccfab5b89664a0e05db"
format_stamp: "Formatted at 2026-08-08 10:34:27 on dist-test-slave-9gcw"
server_key: "4b7c577f9356ac0cd4e2691defa50132"
server_key_iv: "f6e8362b53c2a118edc00aa455893c7f"
server_key_version: "encryptionkey@0"
I20260808 10:34:27.764070 957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "4c120b5093954ccfab5b89664a0e05db"
format_stamp: "Formatted at 2026-08-08 10:34:27 on dist-test-slave-9gcw"
server_key: "4b7c577f9356ac0cd4e2691defa50132"
server_key_iv: "f6e8362b53c2a118edc00aa455893c7f"
server_key_version: "encryptionkey@0"
I20260808 10:34:27.768234 957 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:34:27.770929 973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:27.772265 957 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:34:27.772428 957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "4c120b5093954ccfab5b89664a0e05db"
format_stamp: "Formatted at 2026-08-08 10:34:27 on dist-test-slave-9gcw"
server_key: "4b7c577f9356ac0cd4e2691defa50132"
server_key_iv: "f6e8362b53c2a118edc00aa455893c7f"
server_key_version: "encryptionkey@0"
I20260808 10:34:27.772550 957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:27.813629 957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:27.817529 957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:27.817761 957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:27.819379 957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:27.819473 957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:27.819561 957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:27.819610 957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:34:27 dist-test-slave-9gcw krb5kdc[677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:27.831596 957 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:41841
I20260808 10:34:27.831741 1095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:41841 every 8 connection(s)
I20260808 10:34:27.832659 957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:27.834450 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:27.820364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:43981 (local address 127.30.146.190:44357)
0808 10:34:27.820499 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:27.820503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:27.821654 (+ 1151us) server_negotiation.cc:407] Connection header received
0808 10:34:27.822679 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:27.822685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:27.822760 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:27.822878 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:27.824615 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.825558 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.826938 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.827347 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.827488 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:27.830418 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:27.830423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:27.830426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:27.830462 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:27.832122 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:27.832709 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:27.832713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:27.832715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:27.832843 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:27.833263 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:27.833267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:27.833268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:27.833506 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:27.833604 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:27.834155 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 10:34:27.834265 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:34:27.835340 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:27.820949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:44357 (local address 127.30.146.130:43981)
0808 10:34:27.821496 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:27.821539 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:27.822393 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:27.823167 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:27.823181 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:27.823737 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:27.824431 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:27.824450 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.825708 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.825711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:27.826792 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:27.826802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.827545 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.827549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:27.827814 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:27.828553 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:27.828571 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:27.830229 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:27.832274 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:27.832281 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:27.832296 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:27.832585 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:27.832941 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:27.832944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:27.832946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:27.833087 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:27.833648 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:27.833657 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:27.833908 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 10:34:27.834070 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 10:34:27.834410 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":181,"threads_started":1}
I20260808 10:34:27.836715 1096 heartbeater.cc:347] Connected to a master server at 127.30.146.190:44357
I20260808 10:34:27.836901 1096 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:27.837463 1096 heartbeater.cc:511] Master 127.30.146.190:44357 requested a full tablet report, sending...
I20260808 10:34:27.838624 725 ts_manager.cc:194] Registered new tserver with Master: 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841)
I20260808 10:34:27.839519 725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:43981
I20260808 10:34:27.841182 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 957
I20260808 10:34:27.841375 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:27.841665 31306 external_mini_cluster.cc:1428] Setting key 61567d55b97c8626fec84337c58f2b18
I20260808 10:34:27.848101 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:27.839150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:49745 (local address 127.30.146.190:44357)
0808 10:34:27.839289 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:27.839293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:27.839502 (+ 209us) server_negotiation.cc:407] Connection header received
0808 10:34:27.839576 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:27.839582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:27.839628 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:27.839714 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:27.840261 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.840856 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.842042 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.842330 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.842436 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:27.844140 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:27.844147 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:27.844154 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:27.844188 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:27.845940 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:27.846538 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:27.846543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:27.846548 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:27.846654 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:27.847013 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:27.847019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:27.847021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:27.847357 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:27.847441 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:27.847871 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 10:34:27.847981 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:34:27.848227 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:27.839044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:44357 (local address 127.30.146.130:49745)
0808 10:34:27.839417 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:27.839430 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:27.839511 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:27.839727 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:27.839729 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:27.839982 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:27.840107 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:27.840113 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.841036 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.841040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:27.841915 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:27.841924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:27.842814 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:27.842816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:27.842888 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:27.843504 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:27.843517 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:27.843974 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:27.846055 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:27.846060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:27.846063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:27.846382 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:27.846751 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:27.846776 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:27.846778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:27.846885 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:27.847522 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:27.847528 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:27.847641 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 10:34:27.847756 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:34:27.847935 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:34:27.876588 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:44357
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:27.994858 1105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:27.995153 1105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:27.995239 1105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:27.999482 1105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:27.999583 1105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:27.999652 1105 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:34:27.999698 1105 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:34:27.999768 1105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:28.005326 1105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.146.190:44357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:28.006503 1105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:28.007568 1105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:28.017666 1110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:28.017652 1111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:28.017755 1105 server_base.cc:1034] running on GCE node
W20260808 10:34:28.017652 1113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:28.018440 1105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:28.019129 1105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:28.020329 1105 hybrid_clock.cc:648] HybridClock initialized: now 1786185268020290 us; error 51 us; skew 500 ppm
Aug 08 10:34:28 dist-test-slave-9gcw krb5kdc[677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185268, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:28.023379 1105 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:34:28.024320 1105 webserver.cc:460] Webserver started at http://127.30.146.131:36857/ using document root <none> and password file <none>
I20260808 10:34:28.024856 1105 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:28.024915 1105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:28.025084 1105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:28.027307 1105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "f8bbf88280034cf6a5830591ce141f17"
format_stamp: "Formatted at 2026-08-08 10:34:28 on dist-test-slave-9gcw"
server_key: "d01b130f8dd8358cecb3a06cfeca5b37"
server_key_iv: "0d74f39d7234c2368bbb8c061d5e9766"
server_key_version: "encryptionkey@0"
I20260808 10:34:28.027832 1105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "f8bbf88280034cf6a5830591ce141f17"
format_stamp: "Formatted at 2026-08-08 10:34:28 on dist-test-slave-9gcw"
server_key: "d01b130f8dd8358cecb3a06cfeca5b37"
server_key_iv: "0d74f39d7234c2368bbb8c061d5e9766"
server_key_version: "encryptionkey@0"
I20260808 10:34:28.031921 1105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260808 10:34:28.034673 1121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:28.036020 1105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:34:28.036123 1105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "f8bbf88280034cf6a5830591ce141f17"
format_stamp: "Formatted at 2026-08-08 10:34:28 on dist-test-slave-9gcw"
server_key: "d01b130f8dd8358cecb3a06cfeca5b37"
server_key_iv: "0d74f39d7234c2368bbb8c061d5e9766"
server_key_version: "encryptionkey@0"
I20260808 10:34:28.036216 1105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:28.052066 1105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:28.055779 1105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:28.055953 1105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:28.057466 1105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:28.057530 1105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:28.057616 1105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:28.057657 1105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:34:28 dist-test-slave-9gcw krb5kdc[677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185268, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:28.069756 1105 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:37117
I20260808 10:34:28.069841 1247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:37117 every 8 connection(s)
I20260808 10:34:28.070964 1105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:28.074376 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.058415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:51927 (local address 127.30.146.190:44357)
0808 10:34:28.058595 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.058600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.059544 (+ 944us) server_negotiation.cc:407] Connection header received
0808 10:34:28.060959 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.060964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.061037 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.061129 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:28.063241 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.064157 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.066202 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.066451 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.066544 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.070232 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:28.070235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:28.070237 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:28.070265 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:28.072036 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:28.072631 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:28.072634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:28.072635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:28.072712 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:28.073029 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:28.073031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:28.073032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:28.073296 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:28.073376 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.073975 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.074118 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 10:34:28.074709 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 1105
I20260808 10:34:28.074829 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:28.075189 31306 external_mini_cluster.cc:1428] Setting key fa313925a7f21fa6c6998a46d4e0711d
I20260808 10:34:28.075192 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.058811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:44357 (local address 127.30.146.131:51927)
0808 10:34:28.059331 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.059406 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.060619 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.061708 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.061721 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.062279 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:28.063054 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.063076 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.064289 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.064297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.066046 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.066060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.066689 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.066694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.066935 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.067860 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:28.067885 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:28.070008 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:28.072163 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:28.072168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:28.072180 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:28.072523 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:28.072797 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:28.072800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:28.072801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:28.072936 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:28.073417 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:28.073428 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:28.073663 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.073825 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.074132 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":117,"threads_started":1}
I20260808 10:34:28.076828 1248 heartbeater.cc:347] Connected to a master server at 127.30.146.190:44357
I20260808 10:34:28.076995 1248 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:28.077705 1248 heartbeater.cc:511] Master 127.30.146.190:44357 requested a full tablet report, sending...
I20260808 10:34:28.078830 724 ts_manager.cc:194] Registered new tserver with Master: f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117)
I20260808 10:34:28.079569 724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:51927
I20260808 10:34:28.087894 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.080088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:42407 (local address 127.30.146.190:44357)
0808 10:34:28.080256 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.080260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.080508 (+ 248us) server_negotiation.cc:407] Connection header received
0808 10:34:28.080603 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.080607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.080656 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.080747 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:28.081228 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.082166 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.083073 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.083343 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.083460 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.084567 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:28.084570 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:28.084573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:28.084597 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:28.086158 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:28.086594 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:28.086598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:28.086600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:28.086698 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:28.087038 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:28.087042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:28.087044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:28.087219 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:28.087340 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.087626 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.087736 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:34:28.087915 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.080028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:44357 (local address 127.30.146.131:42407)
0808 10:34:28.080435 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.080448 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.080535 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.080772 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.080774 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.080969 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:28.081068 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.081074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.082285 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.082288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.082930 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.082940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.083476 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.083479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.083573 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.084061 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:28.084072 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:28.084409 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:28.086277 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:28.086282 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:28.086284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:28.086486 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:28.086816 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:28.086819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:28.086821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:28.086938 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:28.087325 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:28.087328 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:28.087425 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.087568 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.087717 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":87,"threads_started":1}
I20260808 10:34:28.089632 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:34:28 dist-test-slave-9gcw krb5kdc[677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:34:28 dist-test-slave-9gcw krb5kdc[677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:28.117547 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.108874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52582 (local address 127.30.146.190:44357)
0808 10:34:28.109010 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.109014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.109025 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:34:28.109065 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.109068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.109108 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.109266 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:28.109973 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.110501 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.111339 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.111552 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.111633 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.114235 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:28.114239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:28.114241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:28.114266 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:28.115729 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:28.116229 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:28.116234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:28.116236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:28.116345 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:28.116632 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:28.116636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:28.116638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:28.116828 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:28.116974 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.117249 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.117371 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:34:28.120790 724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:34:28.124277 724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:28.140190 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.134940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.30.146.131:37117)
0808 10:34:28.135424 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.135439 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.135463 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:34:28.135533 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.135538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.135734 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.135889 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:28.136238 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.137022 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.138766 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.139443 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.139592 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.139746 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.139929 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.140030 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":113,"threads_started":1}
I20260808 10:34:28.140195 1271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.134433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:37117 (local address 127.0.0.1:55214)
0808 10:34:28.135114 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.135128 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.135213 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.135986 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.135989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.136008 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:28.136109 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.136114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.137390 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.137397 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.138621 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.138631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.139564 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.139567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.139652 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.139722 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.139872 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.140064 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":39,"thread_start_us":87,"threads_started":1}
I20260808 10:34:28.140867 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.134080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34031 (local address 127.0.0.1:46322)
0808 10:34:28.134487 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.134501 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.134580 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.135525 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.135532 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.135560 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:28.135796 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.135805 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.137464 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.137467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.139118 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.139130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.140489 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.140493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.140586 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.140613 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.140667 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.140731 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":291,"thread_start_us":94,"threads_started":1}
I20260808 10:34:28.141707 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.134520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.30.146.129:34031)
0808 10:34:28.135002 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.135010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.135034 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:34:28.135111 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.135116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.135352 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.135513 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:28.135933 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.137332 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.139278 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.140149 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.141289 (+ 1140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.141439 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.141490 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.141581 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":89,"threads_started":1}
I20260808 10:34:28.142328 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.133683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41841 (local address 127.0.0.1:42634)
0808 10:34:28.134077 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.134134 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.134267 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.135580 (+ 1313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.135582 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.135594 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:28.135851 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.135863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.137464 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.137467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.139118 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.139130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.141939 (+ 2809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.141943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.142030 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.142053 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.142119 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.142181 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":168,"thread_start_us":99,"threads_started":1}
I20260808 10:34:28.142831 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.133845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42634 (local address 127.30.146.130:41841)
0808 10:34:28.134586 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.134600 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.134627 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:34:28.134708 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.134713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.135408 (+ 695us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.135747 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:28.135952 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.137332 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.139881 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.141196 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.141324 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.142558 (+ 1234us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.142610 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.142696 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":107,"threads_started":1}
I20260808 10:34:28.143750 1177 tablet_service.cc:1467] Processing CreateTablet for tablet b21fc2c6df3d41a29fa7570401aacec9 (DEFAULT_TABLE table=test-table [id=86fa87c767c84fec9e9a05dc66fa449a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:28.144084 865 tablet_service.cc:1467] Processing CreateTablet for tablet b21fc2c6df3d41a29fa7570401aacec9 (DEFAULT_TABLE table=test-table [id=86fa87c767c84fec9e9a05dc66fa449a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:28.144994 1177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21fc2c6df3d41a29fa7570401aacec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:28.145190 865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21fc2c6df3d41a29fa7570401aacec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:28.145486 1025 tablet_service.cc:1467] Processing CreateTablet for tablet b21fc2c6df3d41a29fa7570401aacec9 (DEFAULT_TABLE table=test-table [id=86fa87c767c84fec9e9a05dc66fa449a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:28.146518 1025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21fc2c6df3d41a29fa7570401aacec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:28.151403 1274 tablet_bootstrap.cc:492] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: Bootstrap starting.
I20260808 10:34:28.153214 1275 tablet_bootstrap.cc:492] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: Bootstrap starting.
I20260808 10:34:28.154028 1274 tablet_bootstrap.cc:654] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:28.154556 1276 tablet_bootstrap.cc:492] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: Bootstrap starting.
I20260808 10:34:28.155529 1274 log.cc:824] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:28.155632 1275 tablet_bootstrap.cc:654] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:28.156513 1275 log.cc:824] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:28.157323 1276 tablet_bootstrap.cc:654] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:28.158037 1276 log.cc:824] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:28.158363 1274 tablet_bootstrap.cc:492] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: No bootstrap required, opened a new log
I20260808 10:34:28.158568 1274 ts_tablet_manager.cc:1397] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:34:28.158690 1275 tablet_bootstrap.cc:492] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: No bootstrap required, opened a new log
I20260808 10:34:28.158881 1275 ts_tablet_manager.cc:1397] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:34:28.162981 1276 tablet_bootstrap.cc:492] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: No bootstrap required, opened a new log
I20260808 10:34:28.163233 1276 ts_tablet_manager.cc:1397] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20260808 10:34:28.163141 1275 raft_consensus.cc:348] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } }
I20260808 10:34:28.163419 1275 raft_consensus.cc:374] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:28.163481 1275 raft_consensus.cc:729] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8bbf88280034cf6a5830591ce141f17, State: Initialized, Role: FOLLOWER
I20260808 10:34:28.163848 1274 raft_consensus.cc:348] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } }
I20260808 10:34:28.163955 1275 consensus_queue.cc:260] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } }
I20260808 10:34:28.164094 1274 raft_consensus.cc:374] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:28.164157 1274 raft_consensus.cc:729] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3fa61780cab4954aaa4f257de1bf5be, State: Initialized, Role: FOLLOWER
I20260808 10:34:28.164740 1248 heartbeater.cc:503] Master 127.30.146.190:44357 was elected leader, sending a full tablet report...
I20260808 10:34:28.165136 1275 ts_tablet_manager.cc:1428] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:34:28.165345 1274 consensus_queue.cc:260] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } }
I20260808 10:34:28.166064 947 heartbeater.cc:503] Master 127.30.146.190:44357 was elected leader, sending a full tablet report...
I20260808 10:34:28.166412 1274 ts_tablet_manager.cc:1428] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260808 10:34:28.168141 1276 raft_consensus.cc:348] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } }
I20260808 10:34:28.168895 1276 raft_consensus.cc:374] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:28.168964 1276 raft_consensus.cc:729] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c120b5093954ccfab5b89664a0e05db, State: Initialized, Role: FOLLOWER
I20260808 10:34:28.169406 1276 consensus_queue.cc:260] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } }
I20260808 10:34:28.170379 1276 ts_tablet_manager.cc:1428] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260808 10:34:28.170708 1096 heartbeater.cc:503] Master 127.30.146.190:44357 was elected leader, sending a full tablet report...
I20260808 10:34:28.239226 1281 raft_consensus.cc:482] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:28.239439 1281 raft_consensus.cc:504] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } }
I20260808 10:34:28.240787 1281 leader_election.cc:288] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841), a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031)
I20260808 10:34:28.245178 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.241068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41841 (local address 127.30.146.131:41047)
0808 10:34:28.241295 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.241310 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.241403 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.241860 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.241863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.241879 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:28.242033 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.242042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.243128 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.243131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.244004 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.244013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.244699 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.244703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.244834 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.244853 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.244952 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.245024 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260808 10:34:28.245177 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.241392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:41047 (local address 127.30.146.130:41841)
0808 10:34:28.241541 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.241546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.241562 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:34:28.241618 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.241623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.241696 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.241776 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:28.242169 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.242984 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.244146 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.244565 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.244709 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.244772 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.244972 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.245046 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 10:34:28.246002 1046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21fc2c6df3d41a29fa7570401aacec9" candidate_uuid: "f8bbf88280034cf6a5830591ce141f17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c120b5093954ccfab5b89664a0e05db" is_pre_election: true
I20260808 10:34:28.246171 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.241108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34031 (local address 127.30.146.131:60843)
0808 10:34:28.241614 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.241627 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.241744 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.241993 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.241998 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.242013 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:28.242567 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.242575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.243478 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.243482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.244637 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.244647 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.245794 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.245798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.245893 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.245913 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.245980 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.246039 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":172,"threads_started":1}
I20260808 10:34:28.246232 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.241591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:60843 (local address 127.30.146.129:34031)
0808 10:34:28.241733 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.241737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.241753 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:34:28.241833 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.241836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.241879 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.241954 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:28.242711 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.243350 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.244792 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.245461 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.245586 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.245647 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.246035 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.246120 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:34:28.246326 1046 raft_consensus.cc:2454] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8bbf88280034cf6a5830591ce141f17 in term 0.
I20260808 10:34:28.246862 1123 leader_election.cc:302] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c120b5093954ccfab5b89664a0e05db, f8bbf88280034cf6a5830591ce141f17; no voters:
I20260808 10:34:28.247138 896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21fc2c6df3d41a29fa7570401aacec9" candidate_uuid: "f8bbf88280034cf6a5830591ce141f17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3fa61780cab4954aaa4f257de1bf5be" is_pre_election: true
I20260808 10:34:28.247201 1281 raft_consensus.cc:2781] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:28.247308 1281 raft_consensus.cc:482] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:28.247362 1281 raft_consensus.cc:3037] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:28.247426 896 raft_consensus.cc:2454] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8bbf88280034cf6a5830591ce141f17 in term 0.
I20260808 10:34:28.248557 1281 raft_consensus.cc:504] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } }
I20260808 10:34:28.248955 1281 leader_election.cc:288] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [CANDIDATE]: Term 1 election: Requested vote from peers 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841), a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031)
I20260808 10:34:28.249259 1046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21fc2c6df3d41a29fa7570401aacec9" candidate_uuid: "f8bbf88280034cf6a5830591ce141f17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c120b5093954ccfab5b89664a0e05db"
I20260808 10:34:28.249315 896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21fc2c6df3d41a29fa7570401aacec9" candidate_uuid: "f8bbf88280034cf6a5830591ce141f17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3fa61780cab4954aaa4f257de1bf5be"
I20260808 10:34:28.249383 1046 raft_consensus.cc:3037] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:28.249455 896 raft_consensus.cc:3037] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:28.250478 896 raft_consensus.cc:2454] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8bbf88280034cf6a5830591ce141f17 in term 1.
I20260808 10:34:28.250738 1046 raft_consensus.cc:2454] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8bbf88280034cf6a5830591ce141f17 in term 1.
I20260808 10:34:28.250896 1124 leader_election.cc:302] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3fa61780cab4954aaa4f257de1bf5be, f8bbf88280034cf6a5830591ce141f17; no voters:
I20260808 10:34:28.251099 1281 raft_consensus.cc:2781] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:28.251348 1281 raft_consensus.cc:686] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 1 LEADER]: Becoming Leader. State: Replica: f8bbf88280034cf6a5830591ce141f17, State: Running, Role: LEADER
I20260808 10:34:28.251685 1281 consensus_queue.cc:237] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } }
I20260808 10:34:28.255076 724 catalog_manager.cc:5495] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8bbf88280034cf6a5830591ce141f17 (127.30.146.131). New cstate: current_term: 1 leader_uuid: "f8bbf88280034cf6a5830591ce141f17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } health_report { overall_health: UNKNOWN } } }
Aug 08 10:34:28 dist-test-slave-9gcw krb5kdc[677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:28.316058 1292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.307047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:44357 (local address 127.0.0.1:52594)
0808 10:34:28.307395 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.307408 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.307552 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.307866 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.307869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.308303 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:28.308410 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.308418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.309267 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.309274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.309881 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.309889 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.310404 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.310408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.310522 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.311114 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:28.311126 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:28.312731 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:28.314449 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:28.314458 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:28.314463 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:28.314708 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:28.314983 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:28.314985 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:28.314987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:28.315070 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:28.315478 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:28.315483 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:28.315591 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.315733 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.315873 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":124,"threads_started":1}
I20260808 10:34:28.316062 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.307103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52594 (local address 127.30.146.190:44357)
0808 10:34:28.307290 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.307297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.307497 (+ 200us) server_negotiation.cc:407] Connection header received
0808 10:34:28.307669 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.307672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.307722 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.307820 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:28.308563 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.309121 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.310012 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.310246 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.310383 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.312901 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:28.312904 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:28.312909 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:28.312935 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:28.314328 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:28.314809 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:28.314813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:28.314815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:28.314896 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:28.315155 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:28.315158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:28.315159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:28.315360 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:28.315462 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.315804 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.315937 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":53}
I20260808 10:34:28.319969 724 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:52594:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:34:28.320824 724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260808 10:34:28.322217 1249 tablet.cc:2369] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:34:28.325758 948 tablet.cc:2369] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:34:28.326269 865 tablet_service.cc:1467] Processing CreateTablet for tablet bd69a4bab4834c5ead77a9fd308b8501 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=85bde4961efb485d826f4d80aabb7749]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:34:28.326568 1025 tablet_service.cc:1467] Processing CreateTablet for tablet bd69a4bab4834c5ead77a9fd308b8501 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=85bde4961efb485d826f4d80aabb7749]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:34:28.327100 865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd69a4bab4834c5ead77a9fd308b8501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:28.326763 1177 tablet_service.cc:1467] Processing CreateTablet for tablet bd69a4bab4834c5ead77a9fd308b8501 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=85bde4961efb485d826f4d80aabb7749]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:34:28.327327 1177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd69a4bab4834c5ead77a9fd308b8501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:28.327353 1025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd69a4bab4834c5ead77a9fd308b8501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:28.330080 1274 tablet_bootstrap.cc:492] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: Bootstrap starting.
I20260808 10:34:28.330334 1275 tablet_bootstrap.cc:492] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: Bootstrap starting.
I20260808 10:34:28.330834 1276 tablet_bootstrap.cc:492] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: Bootstrap starting.
I20260808 10:34:28.331156 1274 tablet_bootstrap.cc:654] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:28.331321 1275 tablet_bootstrap.cc:654] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:28.331877 1276 tablet_bootstrap.cc:654] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:28.332795 1275 tablet_bootstrap.cc:492] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: No bootstrap required, opened a new log
I20260808 10:34:28.332897 1275 ts_tablet_manager.cc:1397] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:28.333025 1274 tablet_bootstrap.cc:492] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: No bootstrap required, opened a new log
I20260808 10:34:28.333086 1274 ts_tablet_manager.cc:1397] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:28.333195 1276 tablet_bootstrap.cc:492] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: No bootstrap required, opened a new log
I20260808 10:34:28.333272 1276 ts_tablet_manager.cc:1397] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 10:34:28.333417 1275 raft_consensus.cc:348] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.333540 1275 raft_consensus.cc:374] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:28.333575 1275 raft_consensus.cc:729] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8bbf88280034cf6a5830591ce141f17, State: Initialized, Role: FOLLOWER
I20260808 10:34:28.333614 1274 raft_consensus.cc:348] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.333745 1274 raft_consensus.cc:374] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:28.333712 1275 consensus_queue.cc:260] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.333721 1276 raft_consensus.cc:348] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.333796 1274 raft_consensus.cc:729] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3fa61780cab4954aaa4f257de1bf5be, State: Initialized, Role: FOLLOWER
I20260808 10:34:28.333833 1276 raft_consensus.cc:374] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:28.333868 1276 raft_consensus.cc:729] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c120b5093954ccfab5b89664a0e05db, State: Initialized, Role: FOLLOWER
I20260808 10:34:28.333940 1274 consensus_queue.cc:260] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.333997 1275 ts_tablet_manager.cc:1428] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:34:28.333997 1276 consensus_queue.cc:260] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.334216 1274 ts_tablet_manager.cc:1428] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
W20260808 10:34:28.334234 1097 tablet.cc:2369] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:34:28.334515 1282 tablet_replica.cc:439] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } }
I20260808 10:34:28.334654 1282 tablet_replica.cc:442] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:28.334832 1281 tablet_replica.cc:439] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } }
I20260808 10:34:28.334995 1281 tablet_replica.cc:442] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:28.335384 1276 ts_tablet_manager.cc:1428] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:34:28.335510 1283 tablet_replica.cc:439] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } }
I20260808 10:34:28.335636 1283 tablet_replica.cc:442] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:28.666668 1282 raft_consensus.cc:482] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:28.666841 1282 raft_consensus.cc:504] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.668108 1282 leader_election.cc:288] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841), f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117)
I20260808 10:34:28.672943 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.669057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:38297 (local address 127.30.146.131:37117)
0808 10:34:28.669333 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.669349 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.669398 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:34:28.669624 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.669629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.669683 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.669776 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:28.670066 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.670577 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.671793 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.672304 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.672405 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.672479 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.672709 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.672796 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":84,"threads_started":1}
I20260808 10:34:28.672943 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.668953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:37117 (local address 127.30.146.129:38297)
0808 10:34:28.669331 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.669344 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.669451 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.669837 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.669839 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.669858 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:28.669944 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.669950 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.670739 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.670746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.671650 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.671660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.672453 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.672459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.672560 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.672593 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.672663 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.672753 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":193,"threads_started":1}
I20260808 10:34:28.672976 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.668713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:41699 (local address 127.30.146.130:41841)
0808 10:34:28.668866 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.668873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.668950 (+ 77us) server_negotiation.cc:407] Connection header received
0808 10:34:28.669135 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.669172 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.669230 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.669316 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:28.669819 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.670534 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.671920 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.672387 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.672515 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.672564 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.672774 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.672866 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 10:34:28.672966 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.668470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41841 (local address 127.30.146.129:41699)
0808 10:34:28.668835 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.668852 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.669018 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.669352 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.669356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.669383 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:28.669659 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.669667 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.670817 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.670820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.671781 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.671792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.672532 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.672535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.672610 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.672633 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.672735 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.672800 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":95,"spinlock_wait_cycles":39168,"thread_start_us":103,"threads_started":1}
I20260808 10:34:28.673470 1046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd69a4bab4834c5ead77a9fd308b8501" candidate_uuid: "a3fa61780cab4954aaa4f257de1bf5be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c120b5093954ccfab5b89664a0e05db" is_pre_election: true
I20260808 10:34:28.673666 1046 raft_consensus.cc:2454] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3fa61780cab4954aaa4f257de1bf5be in term 0.
I20260808 10:34:28.673705 1199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd69a4bab4834c5ead77a9fd308b8501" candidate_uuid: "a3fa61780cab4954aaa4f257de1bf5be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bbf88280034cf6a5830591ce141f17" is_pre_election: true
I20260808 10:34:28.673965 1199 raft_consensus.cc:2454] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3fa61780cab4954aaa4f257de1bf5be in term 0.
I20260808 10:34:28.674140 821 leader_election.cc:302] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c120b5093954ccfab5b89664a0e05db, a3fa61780cab4954aaa4f257de1bf5be; no voters:
I20260808 10:34:28.674434 1282 raft_consensus.cc:2781] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:28.674492 1282 raft_consensus.cc:482] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:28.674520 1282 raft_consensus.cc:3037] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:28.675611 1282 raft_consensus.cc:504] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.675967 1282 leader_election.cc:288] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [CANDIDATE]: Term 1 election: Requested vote from peers 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841), f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117)
I20260808 10:34:28.676303 1046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd69a4bab4834c5ead77a9fd308b8501" candidate_uuid: "a3fa61780cab4954aaa4f257de1bf5be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c120b5093954ccfab5b89664a0e05db"
I20260808 10:34:28.676479 1046 raft_consensus.cc:3037] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:28.677649 1046 raft_consensus.cc:2454] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3fa61780cab4954aaa4f257de1bf5be in term 1.
I20260808 10:34:28.677910 1199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd69a4bab4834c5ead77a9fd308b8501" candidate_uuid: "a3fa61780cab4954aaa4f257de1bf5be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bbf88280034cf6a5830591ce141f17"
I20260808 10:34:28.678011 821 leader_election.cc:302] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c120b5093954ccfab5b89664a0e05db, a3fa61780cab4954aaa4f257de1bf5be; no voters:
I20260808 10:34:28.678049 1199 raft_consensus.cc:3037] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:28.678182 1282 raft_consensus.cc:2781] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:28.678416 1282 raft_consensus.cc:686] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [term 1 LEADER]: Becoming Leader. State: Replica: a3fa61780cab4954aaa4f257de1bf5be, State: Running, Role: LEADER
I20260808 10:34:28.678834 1282 consensus_queue.cc:237] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.679262 1199 raft_consensus.cc:2454] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3fa61780cab4954aaa4f257de1bf5be in term 1.
I20260808 10:34:28.681250 1299 tablet_replica.cc:439] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: TxnStatusTablet state changed. Reason: New leader a3fa61780cab4954aaa4f257de1bf5be. Latest consensus state: current_term: 1 leader_uuid: "a3fa61780cab4954aaa4f257de1bf5be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } }
I20260808 10:34:28.681372 1299 tablet_replica.cc:442] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:34:28.681677 1301 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:34:28.681891 1301 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:34:28.682518 725 catalog_manager.cc:5495] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be reported cstate change: term changed from 0 to 1, leader changed from <none> to a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129). New cstate: current_term: 1 leader_uuid: "a3fa61780cab4954aaa4f257de1bf5be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:28.698889 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.694816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.30.146.129:34031)
0808 10:34:28.694947 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.694951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.695046 (+ 95us) server_negotiation.cc:407] Connection header received
0808 10:34:28.695232 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.695234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.695300 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.695397 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:28.695688 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.696176 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.697451 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.697669 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.697749 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.698084 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.698494 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.698581 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.698720 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.698771 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:34:28.698938 1292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.694680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34031 (local address 127.0.0.1:46324)
0808 10:34:28.694959 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.694976 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.695097 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.695449 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.695453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.695462 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:28.695578 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.695585 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.696295 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.696299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.697329 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.697341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.697785 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.697789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.697883 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.697941 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.698602 (+ 661us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.698620 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.698696 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.698792 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260808 10:34:28.704099 1199 raft_consensus.cc:1261] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Refusing update from remote peer a3fa61780cab4954aaa4f257de1bf5be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:28.704123 1046 raft_consensus.cc:1261] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Refusing update from remote peer a3fa61780cab4954aaa4f257de1bf5be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:28.705045 1299 consensus_queue.cc:1035] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:28.705099 1281 consensus_queue.cc:1035] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:28.705443 1282 consensus_queue.cc:1035] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:28.710412 1283 tablet_replica.cc:439] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: TxnStatusTablet state changed. Reason: New leader a3fa61780cab4954aaa4f257de1bf5be. Latest consensus state: current_term: 1 leader_uuid: "a3fa61780cab4954aaa4f257de1bf5be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } }
I20260808 10:34:28.710533 1283 tablet_replica.cc:442] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:28.711015 1285 consensus_queue.cc:1035] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:28.714202 1281 tablet_replica.cc:439] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: TxnStatusTablet state changed. Reason: New leader a3fa61780cab4954aaa4f257de1bf5be. Latest consensus state: current_term: 1 leader_uuid: "a3fa61780cab4954aaa4f257de1bf5be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } }
I20260808 10:34:28.714334 1281 tablet_replica.cc:442] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:28.717856 1299 tablet_replica.cc:439] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3fa61780cab4954aaa4f257de1bf5be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } }
I20260808 10:34:28.717993 1299 tablet_replica.cc:442] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:34:28.718417 1306 mvcc.cc:205] Tried to move back new op lower bound from 7316214860605136896 to 7316214860511707136. Current Snapshot: MvccSnapshot[applied={T|T < 7316214860605136896}]
I20260808 10:34:28.718997 1282 tablet_replica.cc:439] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3fa61780cab4954aaa4f257de1bf5be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } }
I20260808 10:34:28.719105 1282 tablet_replica.cc:442] T bd69a4bab4834c5ead77a9fd308b8501 P a3fa61780cab4954aaa4f257de1bf5be: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:34:28.719777 1315 tablet_replica.cc:439] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3fa61780cab4954aaa4f257de1bf5be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } }
I20260808 10:34:28.719905 1315 tablet_replica.cc:442] T bd69a4bab4834c5ead77a9fd308b8501 P 4c120b5093954ccfab5b89664a0e05db: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:28.720889 1317 tablet_replica.cc:439] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3fa61780cab4954aaa4f257de1bf5be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } } }
I20260808 10:34:28.721019 1317 tablet_replica.cc:442] T bd69a4bab4834c5ead77a9fd308b8501 P f8bbf88280034cf6a5830591ce141f17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:28.721342 1308 mvcc.cc:205] Tried to move back new op lower bound from 7316214860605136896 to 7316214860511707136. Current Snapshot: MvccSnapshot[applied={T|T < 7316214860605136896}]
I20260808 10:34:28.731734 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.727634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55222 (local address 127.30.146.131:37117)
0808 10:34:28.727788 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.727793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.728008 (+ 215us) server_negotiation.cc:407] Connection header received
0808 10:34:28.728289 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.728297 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.728366 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.728486 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:28.728824 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.729410 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.730312 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.730529 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.730642 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.730949 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.731339 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.731472 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.731536 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.731591 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:34:28.743625 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.739978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:48637 (local address 127.30.146.129:34031)
0808 10:34:28.740130 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.740135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.740441 (+ 306us) server_negotiation.cc:407] Connection header received
0808 10:34:28.740557 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.740559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.740615 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.740696 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:28.741023 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.741596 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.742478 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.742709 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.742843 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.743163 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.743243 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.743333 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.743428 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.743479 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:34:28.743637 1327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.739875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34031 (local address 127.30.146.131:48637)
0808 10:34:28.740339 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.740355 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.740464 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.740716 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.740719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.740738 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:28.740904 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.740910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.741711 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.741717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.742345 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.742354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.742855 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.742859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.742952 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.743000 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.743349 (+ 349us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.743357 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.743418 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.743492 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":136,"threads_started":1}
I20260808 10:34:28.774684 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.770605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:36317 (local address 127.30.146.131:37117)
0808 10:34:28.770752 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.770757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.770992 (+ 235us) server_negotiation.cc:407] Connection header received
0808 10:34:28.771130 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.771132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.771177 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.771300 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:28.771680 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.772212 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.773443 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.773713 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.773853 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.774189 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.774270 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.774359 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.774452 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.774504 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:34:28.774729 1330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.770518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:37117 (local address 127.30.146.129:36317)
0808 10:34:28.770890 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.770906 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.771031 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.771336 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.771339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.771347 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:28.771498 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.771504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.772343 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.772349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.773239 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.773252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.773869 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.773873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.773995 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.774043 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.774366 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.774375 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.774441 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.774522 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":153,"threads_started":1}
I20260808 10:34:28.806173 725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:28.806447 725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:28.809785 725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f: Sending DeleteTablet for 3 replicas of tablet b21fc2c6df3d41a29fa7570401aacec9
I20260808 10:34:28.810626 1025 tablet_service.cc:1514] Processing DeleteTablet for tablet b21fc2c6df3d41a29fa7570401aacec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:28 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:42634
I20260808 10:34:28.810842 865 tablet_service.cc:1514] Processing DeleteTablet for tablet b21fc2c6df3d41a29fa7570401aacec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:28 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:46322
I20260808 10:34:28.811363 1334 tablet_replica.cc:330] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: stopping tablet replica
I20260808 10:34:28.811589 1177 tablet_service.cc:1514] Processing DeleteTablet for tablet b21fc2c6df3d41a29fa7570401aacec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:28 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:55214
I20260808 10:34:28.811761 1334 raft_consensus.cc:2229] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:28.811959 1334 raft_consensus.cc:2258] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:28.812955 725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:34:28.813549 1334 ts_tablet_manager.cc:1905] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:34:28.813870 725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:28.815795 1335 tablet_replica.cc:330] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: stopping tablet replica
I20260808 10:34:28.816167 1335 raft_consensus.cc:2229] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:34:28.819015 1334 ts_tablet_manager.cc:1918] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:34:28.819159 1334 log.cc:1196] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/b21fc2c6df3d41a29fa7570401aacec9
I20260808 10:34:28.819540 1334 ts_tablet_manager.cc:1939] T b21fc2c6df3d41a29fa7570401aacec9 P a3fa61780cab4954aaa4f257de1bf5be: Deleting consensus metadata
I20260808 10:34:28.820797 865 tablet_service.cc:1467] Processing CreateTablet for tablet 950f83b9a32d4cd8baf2c0780b0262ab (DEFAULT_TABLE table=test-table [id=ea7a4cd2c19f4143a7ca38ab46a66cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:28.820917 711 catalog_manager.cc:4841] TS a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031): tablet b21fc2c6df3d41a29fa7570401aacec9 (table test-table [id=86fa87c767c84fec9e9a05dc66fa449a]) successfully deleted
I20260808 10:34:28.821064 865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 950f83b9a32d4cd8baf2c0780b0262ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:28.822044 1177 tablet_service.cc:1467] Processing CreateTablet for tablet 950f83b9a32d4cd8baf2c0780b0262ab (DEFAULT_TABLE table=test-table [id=ea7a4cd2c19f4143a7ca38ab46a66cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:28.822086 1025 tablet_service.cc:1467] Processing CreateTablet for tablet 950f83b9a32d4cd8baf2c0780b0262ab (DEFAULT_TABLE table=test-table [id=ea7a4cd2c19f4143a7ca38ab46a66cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:28.822367 1177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 950f83b9a32d4cd8baf2c0780b0262ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:28.822381 1025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 950f83b9a32d4cd8baf2c0780b0262ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:28.824913 1276 tablet_bootstrap.cc:492] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db: Bootstrap starting.
I20260808 10:34:28.825394 1275 tablet_bootstrap.cc:492] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17: Bootstrap starting.
I20260808 10:34:28.826005 1276 tablet_bootstrap.cc:654] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:28.826452 1275 tablet_bootstrap.cc:654] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:28.828063 1275 tablet_bootstrap.cc:492] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17: No bootstrap required, opened a new log
I20260808 10:34:28.828161 1275 ts_tablet_manager.cc:1397] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:28.828691 1275 raft_consensus.cc:348] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.828823 1275 raft_consensus.cc:374] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:28.828858 1275 raft_consensus.cc:729] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8bbf88280034cf6a5830591ce141f17, State: Initialized, Role: FOLLOWER
I20260808 10:34:28.829003 1275 consensus_queue.cc:260] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.829319 1275 ts_tablet_manager.cc:1428] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:34:28.829391 1274 tablet_bootstrap.cc:492] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be: Bootstrap starting.
I20260808 10:34:28.830431 1274 tablet_bootstrap.cc:654] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:28.831898 1335 raft_consensus.cc:2258] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:28.832177 1333 tablet_replica.cc:330] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: stopping tablet replica
I20260808 10:34:28.833956 1333 raft_consensus.cc:2229] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:28.834864 1335 ts_tablet_manager.cc:1905] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:28.835564 1333 raft_consensus.cc:2258] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:28.836810 1274 tablet_bootstrap.cc:492] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be: No bootstrap required, opened a new log
I20260808 10:34:28.836918 1274 ts_tablet_manager.cc:1397] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260808 10:34:28.837487 1274 raft_consensus.cc:348] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.837625 1274 raft_consensus.cc:374] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:28.837666 1274 raft_consensus.cc:729] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3fa61780cab4954aaa4f257de1bf5be, State: Initialized, Role: FOLLOWER
I20260808 10:34:28.837863 1333 ts_tablet_manager.cc:1905] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:28.837810 1274 consensus_queue.cc:260] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.838091 1274 ts_tablet_manager.cc:1428] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260808 10:34:28.841436 1276 tablet_bootstrap.cc:492] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db: No bootstrap required, opened a new log
I20260808 10:34:28.841531 1276 ts_tablet_manager.cc:1397] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260808 10:34:28.842000 1276 raft_consensus.cc:348] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.842108 1276 raft_consensus.cc:374] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:28.842139 1276 raft_consensus.cc:729] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c120b5093954ccfab5b89664a0e05db, State: Initialized, Role: FOLLOWER
I20260808 10:34:28.842254 1276 consensus_queue.cc:260] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.842504 1276 ts_tablet_manager.cc:1428] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:28.846720 1335 ts_tablet_manager.cc:1918] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:34:28.849860 1333 ts_tablet_manager.cc:1918] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:34:28.850104 1335 log.cc:1196] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/b21fc2c6df3d41a29fa7570401aacec9
I20260808 10:34:28.850572 1335 ts_tablet_manager.cc:1939] T b21fc2c6df3d41a29fa7570401aacec9 P f8bbf88280034cf6a5830591ce141f17: Deleting consensus metadata
I20260808 10:34:28.850721 1333 log.cc:1196] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/b21fc2c6df3d41a29fa7570401aacec9
I20260808 10:34:28.852275 709 catalog_manager.cc:4841] TS f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117): tablet b21fc2c6df3d41a29fa7570401aacec9 (table test-table [id=86fa87c767c84fec9e9a05dc66fa449a]) successfully deleted
I20260808 10:34:28.853943 1333 ts_tablet_manager.cc:1939] T b21fc2c6df3d41a29fa7570401aacec9 P 4c120b5093954ccfab5b89664a0e05db: Deleting consensus metadata
I20260808 10:34:28.855419 709 catalog_manager.cc:4841] TS 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841): tablet b21fc2c6df3d41a29fa7570401aacec9 (table test-table [id=86fa87c767c84fec9e9a05dc66fa449a]) successfully deleted
I20260808 10:34:28.879334 1315 raft_consensus.cc:482] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:28.879567 1315 raft_consensus.cc:504] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.881551 1315 leader_election.cc:288] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031), f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117)
I20260808 10:34:28.888327 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.882314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:52403 (local address 127.30.146.131:37117)
0808 10:34:28.882444 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.882449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.882720 (+ 271us) server_negotiation.cc:407] Connection header received
0808 10:34:28.882914 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.882917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.882975 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.883024 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:28.883490 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.884525 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.886458 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.887110 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.887208 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.887268 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.887955 (+ 687us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.888075 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:34:28.889051 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.881948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:37117 (local address 127.30.146.130:52403)
0808 10:34:28.882639 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.882653 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.882803 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.883158 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.883161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.883175 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:28.883374 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.883380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.884875 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.884879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.886289 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.886301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.887512 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.887517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.887780 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.887811 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.888748 (+ 937us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.888839 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":59,"thread_start_us":73,"threads_started":1}
I20260808 10:34:28.889174 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.882138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:60871 (local address 127.30.146.129:34031)
0808 10:34:28.882268 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.882273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.882599 (+ 326us) server_negotiation.cc:407] Connection header received
0808 10:34:28.882803 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.882806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.882858 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.882904 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:28.883594 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.884456 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.887337 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.888086 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.888215 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.888289 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.888881 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.889023 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:34:28.889781 1199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950f83b9a32d4cd8baf2c0780b0262ab" candidate_uuid: "4c120b5093954ccfab5b89664a0e05db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bbf88280034cf6a5830591ce141f17" is_pre_election: true
I20260808 10:34:28.889981 1199 raft_consensus.cc:2454] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c120b5093954ccfab5b89664a0e05db in term 0.
I20260808 10:34:28.890022 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.881779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34031 (local address 127.30.146.130:60871)
0808 10:34:28.882508 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.882523 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.882679 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.883128 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.883131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.883167 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:28.883436 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.883443 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.884606 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.884609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.887178 (+ 2569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.887189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.888630 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.888635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.888731 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.888754 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.889691 (+ 937us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.889775 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":73,"thread_start_us":90,"threads_started":1}
I20260808 10:34:28.890645 896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950f83b9a32d4cd8baf2c0780b0262ab" candidate_uuid: "4c120b5093954ccfab5b89664a0e05db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3fa61780cab4954aaa4f257de1bf5be" is_pre_election: true
I20260808 10:34:28.890785 896 raft_consensus.cc:2454] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c120b5093954ccfab5b89664a0e05db in term 0.
I20260808 10:34:28.891216 975 leader_election.cc:302] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c120b5093954ccfab5b89664a0e05db, f8bbf88280034cf6a5830591ce141f17; no voters:
I20260808 10:34:28.891595 1315 raft_consensus.cc:2781] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:28.891669 1315 raft_consensus.cc:482] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:28.891710 1315 raft_consensus.cc:3037] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:28.894558 1315 raft_consensus.cc:504] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.894989 1315 leader_election.cc:288] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [CANDIDATE]: Term 1 election: Requested vote from peers a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031), f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117)
I20260808 10:34:28.895467 896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950f83b9a32d4cd8baf2c0780b0262ab" candidate_uuid: "4c120b5093954ccfab5b89664a0e05db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3fa61780cab4954aaa4f257de1bf5be"
I20260808 10:34:28.895594 896 raft_consensus.cc:3037] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:28.895604 1199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950f83b9a32d4cd8baf2c0780b0262ab" candidate_uuid: "4c120b5093954ccfab5b89664a0e05db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bbf88280034cf6a5830591ce141f17"
I20260808 10:34:28.895741 1199 raft_consensus.cc:3037] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:28.896878 896 raft_consensus.cc:2454] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c120b5093954ccfab5b89664a0e05db in term 1.
I20260808 10:34:28.897315 1199 raft_consensus.cc:2454] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c120b5093954ccfab5b89664a0e05db in term 1.
I20260808 10:34:28.897384 976 leader_election.cc:302] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c120b5093954ccfab5b89664a0e05db, a3fa61780cab4954aaa4f257de1bf5be; no voters:
I20260808 10:34:28.897912 1315 raft_consensus.cc:2781] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:28.898197 1315 raft_consensus.cc:686] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 1 LEADER]: Becoming Leader. State: Replica: 4c120b5093954ccfab5b89664a0e05db, State: Running, Role: LEADER
I20260808 10:34:28.898545 1315 consensus_queue.cc:237] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:28.901714 724 catalog_manager.cc:5495] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c120b5093954ccfab5b89664a0e05db (127.30.146.130). New cstate: current_term: 1 leader_uuid: "4c120b5093954ccfab5b89664a0e05db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:28.930964 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.925595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42640 (local address 127.30.146.130:41841)
0808 10:34:28.926076 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.926082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.926097 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:34:28.926176 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.926179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.926244 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.926356 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:28.926791 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.927648 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.928906 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.929326 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.929490 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.929832 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.930440 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.930564 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.930719 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.930818 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387}
I20260808 10:34:28.952081 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.947790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:55149 (local address 127.30.146.129:34031)
0808 10:34:28.947975 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.947980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.948196 (+ 216us) server_negotiation.cc:407] Connection header received
0808 10:34:28.948385 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.948388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.948446 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.948531 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:28.948982 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.949683 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.950795 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.951045 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.951160 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.951557 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.951649 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.951730 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.951857 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.951916 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 10:34:28.952150 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.947574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34031 (local address 127.30.146.130:55149)
0808 10:34:28.948061 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.948074 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.948259 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.948559 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.948562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.948570 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:28.948838 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.948846 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.949822 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.949826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.950650 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.950659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.951185 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.951189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.951339 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.951390 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.951750 (+ 360us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.951757 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.951843 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.951934 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":176,"threads_started":1}
I20260808 10:34:28.960906 1199 raft_consensus.cc:1261] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Refusing update from remote peer 4c120b5093954ccfab5b89664a0e05db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:28.961542 1283 consensus_queue.cc:1035] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:28.961957 896 raft_consensus.cc:1261] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Refusing update from remote peer 4c120b5093954ccfab5b89664a0e05db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:28.962554 1283 consensus_queue.cc:1035] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:28.985808 1330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.981442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41841 (local address 127.30.146.129:46027)
0808 10:34:28.981699 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:28.981727 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:28.981808 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:28.982418 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:28.982422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:28.982430 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:28.982538 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.982545 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.983483 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.983489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.984389 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:28.984400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.984948 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.984951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:28.985037 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.985076 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.985487 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.985496 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:34:28.985567 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:34:28.985655 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260808 10:34:28.985831 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:28.981930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:46027 (local address 127.30.146.130:41841)
0808 10:34:28.982084 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:28.982089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:28.982104 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:34:28.982158 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:28.982161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:28.982211 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:28.982294 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:28.982651 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.983370 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.984529 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:28.984824 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:28.984950 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:28.985297 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:28.985375 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:28.985481 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:28.985627 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 10:34:28.985694 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 10:34:28.986699 1314 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 10:34:30.560038 723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:30.560204 723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:30.563067 723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f: Sending DeleteTablet for 3 replicas of tablet 950f83b9a32d4cd8baf2c0780b0262ab
I20260808 10:34:30.563772 1025 tablet_service.cc:1514] Processing DeleteTablet for tablet 950f83b9a32d4cd8baf2c0780b0262ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:30 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:42634
I20260808 10:34:30.563911 865 tablet_service.cc:1514] Processing DeleteTablet for tablet 950f83b9a32d4cd8baf2c0780b0262ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:30 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:46322
I20260808 10:34:30.564349 1377 tablet_replica.cc:330] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db: stopping tablet replica
I20260808 10:34:30.564500 1377 raft_consensus.cc:2229] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:34:30.564844 1377 raft_consensus.cc:2258] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:30.564925 1378 tablet_replica.cc:330] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be: stopping tablet replica
I20260808 10:34:30.565076 1378 raft_consensus.cc:2229] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:30.565245 1378 raft_consensus.cc:2258] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:30.565636 1377 ts_tablet_manager.cc:1905] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:30.565935 1378 ts_tablet_manager.cc:1905] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:30.566706 1177 tablet_service.cc:1514] Processing DeleteTablet for tablet 950f83b9a32d4cd8baf2c0780b0262ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:30 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:55214
I20260808 10:34:30.568857 1378 ts_tablet_manager.cc:1918] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:34:30.568857 1377 ts_tablet_manager.cc:1918] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:34:30.568948 1378 log.cc:1196] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/950f83b9a32d4cd8baf2c0780b0262ab
I20260808 10:34:30.568948 1377 log.cc:1196] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/950f83b9a32d4cd8baf2c0780b0262ab
I20260808 10:34:30.569267 1378 ts_tablet_manager.cc:1939] T 950f83b9a32d4cd8baf2c0780b0262ab P a3fa61780cab4954aaa4f257de1bf5be: Deleting consensus metadata
I20260808 10:34:30.569465 1377 ts_tablet_manager.cc:1939] T 950f83b9a32d4cd8baf2c0780b0262ab P 4c120b5093954ccfab5b89664a0e05db: Deleting consensus metadata
I20260808 10:34:30.569639 1380 tablet_replica.cc:330] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17: stopping tablet replica
I20260808 10:34:30.569861 1380 raft_consensus.cc:2229] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:30.570061 1380 raft_consensus.cc:2258] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:30.570381 709 catalog_manager.cc:4841] TS 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841): tablet 950f83b9a32d4cd8baf2c0780b0262ab (table test-table [id=ea7a4cd2c19f4143a7ca38ab46a66cec]) successfully deleted
I20260808 10:34:30.570925 1380 ts_tablet_manager.cc:1905] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:30.570897 711 catalog_manager.cc:4841] TS a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031): tablet 950f83b9a32d4cd8baf2c0780b0262ab (table test-table [id=ea7a4cd2c19f4143a7ca38ab46a66cec]) successfully deleted
I20260808 10:34:30.573801 1380 ts_tablet_manager.cc:1918] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:34:30.573908 1380 log.cc:1196] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/950f83b9a32d4cd8baf2c0780b0262ab
I20260808 10:34:30.574198 1380 ts_tablet_manager.cc:1939] T 950f83b9a32d4cd8baf2c0780b0262ab P f8bbf88280034cf6a5830591ce141f17: Deleting consensus metadata
I20260808 10:34:30.575106 709 catalog_manager.cc:4841] TS f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117): tablet 950f83b9a32d4cd8baf2c0780b0262ab (table test-table [id=ea7a4cd2c19f4143a7ca38ab46a66cec]) successfully deleted
Aug 08 10:34:30 dist-test-slave-9gcw krb5kdc[677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185270, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:34:30 dist-test-slave-9gcw krb5kdc[677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185270, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:30.598665 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:30.588987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.30.146.190:44357)
0808 10:34:30.589402 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:30.589411 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:30.589426 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:34:30.589541 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:30.589544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:30.589598 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:30.589697 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:30.590438 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:30.591008 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:30.592000 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:30.592239 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:30.592349 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:30.595088 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:30.595091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:30.595093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:30.595121 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:30.596856 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:30.597365 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:30.597368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:30.597369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:30.597451 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:30.597860 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:30.597862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:30.597863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:30.598053 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:30.598158 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:30.598418 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 10:34:30.598530 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":100,"threads_started":1}
I20260808 10:34:30.601868 723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:34:30.602378 723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:30.608219 1025 tablet_service.cc:1467] Processing CreateTablet for tablet 41f17e21327647808475e1732aac9f15 (DEFAULT_TABLE table=test-table [id=016f7c04ad7247fda7073ae794767fdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:30.608219 1177 tablet_service.cc:1467] Processing CreateTablet for tablet 41f17e21327647808475e1732aac9f15 (DEFAULT_TABLE table=test-table [id=016f7c04ad7247fda7073ae794767fdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:30.608274 865 tablet_service.cc:1467] Processing CreateTablet for tablet 41f17e21327647808475e1732aac9f15 (DEFAULT_TABLE table=test-table [id=016f7c04ad7247fda7073ae794767fdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:30.608788 1177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f17e21327647808475e1732aac9f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:30.608788 1025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f17e21327647808475e1732aac9f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:30.608831 865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f17e21327647808475e1732aac9f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:30.611505 1390 tablet_bootstrap.cc:492] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be: Bootstrap starting.
I20260808 10:34:30.611644 1388 tablet_bootstrap.cc:492] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db: Bootstrap starting.
I20260808 10:34:30.612495 1388 tablet_bootstrap.cc:654] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:30.612500 1390 tablet_bootstrap.cc:654] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:30.613222 1389 tablet_bootstrap.cc:492] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17: Bootstrap starting.
I20260808 10:34:30.613891 1388 tablet_bootstrap.cc:492] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db: No bootstrap required, opened a new log
I20260808 10:34:30.614044 1388 ts_tablet_manager.cc:1397] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:34:30.614166 1389 tablet_bootstrap.cc:654] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:30.614176 1390 tablet_bootstrap.cc:492] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be: No bootstrap required, opened a new log
I20260808 10:34:30.614284 1390 ts_tablet_manager.cc:1397] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:30.614571 1388 raft_consensus.cc:348] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:30.614715 1388 raft_consensus.cc:374] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:30.614760 1388 raft_consensus.cc:729] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c120b5093954ccfab5b89664a0e05db, State: Initialized, Role: FOLLOWER
I20260808 10:34:30.614712 1390 raft_consensus.cc:348] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:30.614861 1390 raft_consensus.cc:374] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:30.614910 1390 raft_consensus.cc:729] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3fa61780cab4954aaa4f257de1bf5be, State: Initialized, Role: FOLLOWER
I20260808 10:34:30.614988 1388 consensus_queue.cc:260] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:30.615067 1390 consensus_queue.cc:260] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:30.615363 1388 ts_tablet_manager.cc:1428] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:30.615707 1389 tablet_bootstrap.cc:492] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17: No bootstrap required, opened a new log
I20260808 10:34:30.615814 1389 ts_tablet_manager.cc:1397] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:30.616245 1389 raft_consensus.cc:348] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:30.616362 1389 raft_consensus.cc:374] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:30.616396 1389 raft_consensus.cc:729] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8bbf88280034cf6a5830591ce141f17, State: Initialized, Role: FOLLOWER
I20260808 10:34:30.616575 1389 consensus_queue.cc:260] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:30.617012 1389 ts_tablet_manager.cc:1428] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:30.617030 1390 ts_tablet_manager.cc:1428] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 10:34:30.872346 1353 raft_consensus.cc:482] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:30.872565 1353 raft_consensus.cc:504] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:30.873127 1353 leader_election.cc:288] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841), f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117)
I20260808 10:34:30.873732 1046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f17e21327647808475e1732aac9f15" candidate_uuid: "a3fa61780cab4954aaa4f257de1bf5be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c120b5093954ccfab5b89664a0e05db" is_pre_election: true
I20260808 10:34:30.873828 1199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f17e21327647808475e1732aac9f15" candidate_uuid: "a3fa61780cab4954aaa4f257de1bf5be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bbf88280034cf6a5830591ce141f17" is_pre_election: true
I20260808 10:34:30.873917 1046 raft_consensus.cc:2454] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3fa61780cab4954aaa4f257de1bf5be in term 0.
I20260808 10:34:30.874013 1199 raft_consensus.cc:2454] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3fa61780cab4954aaa4f257de1bf5be in term 0.
I20260808 10:34:30.874321 821 leader_election.cc:302] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c120b5093954ccfab5b89664a0e05db, a3fa61780cab4954aaa4f257de1bf5be; no voters:
I20260808 10:34:30.874502 1353 raft_consensus.cc:2781] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:30.874565 1353 raft_consensus.cc:482] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:30.874614 1353 raft_consensus.cc:3037] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:30.875759 1353 raft_consensus.cc:504] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:30.876134 1353 leader_election.cc:288] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [CANDIDATE]: Term 1 election: Requested vote from peers 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841), f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117)
I20260808 10:34:30.876530 1199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f17e21327647808475e1732aac9f15" candidate_uuid: "a3fa61780cab4954aaa4f257de1bf5be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bbf88280034cf6a5830591ce141f17"
I20260808 10:34:30.876533 1046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f17e21327647808475e1732aac9f15" candidate_uuid: "a3fa61780cab4954aaa4f257de1bf5be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c120b5093954ccfab5b89664a0e05db"
I20260808 10:34:30.876677 1199 raft_consensus.cc:3037] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:30.876688 1046 raft_consensus.cc:3037] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:30.877807 1046 raft_consensus.cc:2454] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3fa61780cab4954aaa4f257de1bf5be in term 1.
I20260808 10:34:30.877815 1199 raft_consensus.cc:2454] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3fa61780cab4954aaa4f257de1bf5be in term 1.
I20260808 10:34:30.878109 821 leader_election.cc:302] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c120b5093954ccfab5b89664a0e05db, a3fa61780cab4954aaa4f257de1bf5be; no voters:
I20260808 10:34:30.878353 1353 raft_consensus.cc:2781] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:30.878494 1353 raft_consensus.cc:686] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 1 LEADER]: Becoming Leader. State: Replica: a3fa61780cab4954aaa4f257de1bf5be, State: Running, Role: LEADER
I20260808 10:34:30.878676 1353 consensus_queue.cc:237] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:30.880705 722 catalog_manager.cc:5495] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be reported cstate change: term changed from 0 to 1, leader changed from <none> to a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129). New cstate: current_term: 1 leader_uuid: "a3fa61780cab4954aaa4f257de1bf5be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:30.919843 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:30.916347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46330 (local address 127.30.146.129:34031)
0808 10:34:30.916645 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:30.916653 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:30.916665 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:34:30.916705 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:30.916708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:30.916750 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:30.916826 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:30.917128 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:30.917722 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:30.918680 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:30.919016 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:30.919119 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:30.919403 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:30.919475 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:30.919531 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:30.919654 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 10:34:30.919705 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":142,"threads_started":1}
I20260808 10:34:30.931459 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:30.926796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34031 (local address 127.30.146.129:45301)
0808 10:34:30.927119 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:30.927145 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:30.927271 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:30.927886 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:30.927889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:30.927896 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:30.928022 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:30.928031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:30.928859 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:30.928863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:30.929976 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:30.929988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:30.930643 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:30.930646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:30.930767 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:30.930791 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:30.931143 (+ 352us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:30.931150 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:34:30.931209 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:34:30.931317 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":50,"thread_start_us":125,"threads_started":1}
I20260808 10:34:30.931458 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:30.927055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:45301 (local address 127.30.146.129:34031)
0808 10:34:30.927649 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:30.927654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:30.927668 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:34:30.927731 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:30.927733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:30.927772 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:30.927863 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:30.928179 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:30.928718 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:30.930138 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:30.930493 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:30.930634 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:30.930947 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:30.931038 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:30.931129 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:30.931216 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 10:34:30.931291 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":508}
I20260808 10:34:30.938359 1046 raft_consensus.cc:1261] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Refusing update from remote peer a3fa61780cab4954aaa4f257de1bf5be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:30.938787 1199 raft_consensus.cc:1261] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Refusing update from remote peer a3fa61780cab4954aaa4f257de1bf5be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:30.938853 1356 consensus_queue.cc:1035] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:30.939175 1353 consensus_queue.cc:1035] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:30.944056 1358 mvcc.cc:205] Tried to move back new op lower bound from 7316214869759225856 to 7316214869520642048. Current Snapshot: MvccSnapshot[applied={T|T < 7316214869759225856}]
I20260808 10:34:30.978112 725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:30.978294 725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:30.980872 725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f: Sending DeleteTablet for 3 replicas of tablet 41f17e21327647808475e1732aac9f15
I20260808 10:34:30.981530 873 tablet_service.cc:1514] Processing DeleteTablet for tablet 41f17e21327647808475e1732aac9f15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:30 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:46322
I20260808 10:34:30.981530 1025 tablet_service.cc:1514] Processing DeleteTablet for tablet 41f17e21327647808475e1732aac9f15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:30 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:42634
I20260808 10:34:30.981760 1377 tablet_replica.cc:330] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db: stopping tablet replica
I20260808 10:34:30.981766 1177 tablet_service.cc:1514] Processing DeleteTablet for tablet 41f17e21327647808475e1732aac9f15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:30 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:55214
I20260808 10:34:30.981899 1377 raft_consensus.cc:2229] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:30.981976 1380 tablet_replica.cc:330] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17: stopping tablet replica
I20260808 10:34:30.982015 1377 raft_consensus.cc:2258] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:30.982137 1380 raft_consensus.cc:2229] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:30.982237 1378 tablet_replica.cc:330] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be: stopping tablet replica
I20260808 10:34:30.982314 1380 raft_consensus.cc:2258] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:30.982388 1378 raft_consensus.cc:2229] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:34:30.982822 1378 raft_consensus.cc:2258] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:30.983089 1380 ts_tablet_manager.cc:1905] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:30.983640 1377 ts_tablet_manager.cc:1905] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:30.985914 1380 ts_tablet_manager.cc:1918] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:34:30.985996 1380 log.cc:1196] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/41f17e21327647808475e1732aac9f15
I20260808 10:34:30.985971 1377 ts_tablet_manager.cc:1918] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:34:30.986049 1377 log.cc:1196] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/41f17e21327647808475e1732aac9f15
I20260808 10:34:30.986272 1380 ts_tablet_manager.cc:1939] T 41f17e21327647808475e1732aac9f15 P f8bbf88280034cf6a5830591ce141f17: Deleting consensus metadata
I20260808 10:34:30.986287 1378 ts_tablet_manager.cc:1905] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:30.986636 1377 ts_tablet_manager.cc:1939] T 41f17e21327647808475e1732aac9f15 P 4c120b5093954ccfab5b89664a0e05db: Deleting consensus metadata
I20260808 10:34:30.987213 709 catalog_manager.cc:4841] TS f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117): tablet 41f17e21327647808475e1732aac9f15 (table test-table [id=016f7c04ad7247fda7073ae794767fdd]) successfully deleted
I20260808 10:34:30.988897 1378 ts_tablet_manager.cc:1918] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:34:30.988996 1378 log.cc:1196] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/41f17e21327647808475e1732aac9f15
I20260808 10:34:30.989326 1378 ts_tablet_manager.cc:1939] T 41f17e21327647808475e1732aac9f15 P a3fa61780cab4954aaa4f257de1bf5be: Deleting consensus metadata
I20260808 10:34:30.989456 709 catalog_manager.cc:4841] TS 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841): tablet 41f17e21327647808475e1732aac9f15 (table test-table [id=016f7c04ad7247fda7073ae794767fdd]) successfully deleted
I20260808 10:34:30.990017 711 catalog_manager.cc:4841] TS a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031): tablet 41f17e21327647808475e1732aac9f15 (table test-table [id=016f7c04ad7247fda7073ae794767fdd]) successfully deleted
I20260808 10:34:30.991037 725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:34:30.991488 725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:30.996454 873 tablet_service.cc:1467] Processing CreateTablet for tablet a4769f6d36004a5ebed50495c422e951 (DEFAULT_TABLE table=test-table [id=0cb84472265c4373a8ceb5841d9cbcec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:30.996649 1025 tablet_service.cc:1467] Processing CreateTablet for tablet a4769f6d36004a5ebed50495c422e951 (DEFAULT_TABLE table=test-table [id=0cb84472265c4373a8ceb5841d9cbcec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:30.996794 873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4769f6d36004a5ebed50495c422e951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:30.996740 1177 tablet_service.cc:1467] Processing CreateTablet for tablet a4769f6d36004a5ebed50495c422e951 (DEFAULT_TABLE table=test-table [id=0cb84472265c4373a8ceb5841d9cbcec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:30.996914 1025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4769f6d36004a5ebed50495c422e951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:30.997009 1177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4769f6d36004a5ebed50495c422e951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:30.999523 1389 tablet_bootstrap.cc:492] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17: Bootstrap starting.
I20260808 10:34:30.999620 1388 tablet_bootstrap.cc:492] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db: Bootstrap starting.
I20260808 10:34:30.999631 1390 tablet_bootstrap.cc:492] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be: Bootstrap starting.
I20260808 10:34:31.000479 1388 tablet_bootstrap.cc:654] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:31.000479 1389 tablet_bootstrap.cc:654] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:31.000586 1390 tablet_bootstrap.cc:654] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:31.002421 1388 tablet_bootstrap.cc:492] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db: No bootstrap required, opened a new log
I20260808 10:34:31.002547 1388 ts_tablet_manager.cc:1397] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:31.002951 1388 raft_consensus.cc:348] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:31.003086 1388 raft_consensus.cc:374] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:31.003139 1388 raft_consensus.cc:729] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c120b5093954ccfab5b89664a0e05db, State: Initialized, Role: FOLLOWER
I20260808 10:34:31.003298 1388 consensus_queue.cc:260] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:31.003593 1388 ts_tablet_manager.cc:1428] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:34:31.004503 1389 tablet_bootstrap.cc:492] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17: No bootstrap required, opened a new log
I20260808 10:34:31.004616 1389 ts_tablet_manager.cc:1397] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 10:34:31.005275 1389 raft_consensus.cc:348] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:31.005419 1389 raft_consensus.cc:374] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:31.005458 1389 raft_consensus.cc:729] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8bbf88280034cf6a5830591ce141f17, State: Initialized, Role: FOLLOWER
I20260808 10:34:31.005578 1389 consensus_queue.cc:260] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:31.005914 1390 tablet_bootstrap.cc:492] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be: No bootstrap required, opened a new log
I20260808 10:34:31.005931 1389 ts_tablet_manager.cc:1428] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:31.005992 1390 ts_tablet_manager.cc:1397] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 10:34:31.006553 1390 raft_consensus.cc:348] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:31.006717 1390 raft_consensus.cc:374] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:31.006753 1390 raft_consensus.cc:729] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3fa61780cab4954aaa4f257de1bf5be, State: Initialized, Role: FOLLOWER
I20260808 10:34:31.006886 1390 consensus_queue.cc:260] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:31.007184 1390 ts_tablet_manager.cc:1428] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:31.013621 1367 raft_consensus.cc:482] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:31.013737 1367 raft_consensus.cc:504] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:31.014204 1367 leader_election.cc:288] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031), f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117)
I20260808 10:34:31.014611 896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4769f6d36004a5ebed50495c422e951" candidate_uuid: "4c120b5093954ccfab5b89664a0e05db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3fa61780cab4954aaa4f257de1bf5be" is_pre_election: true
I20260808 10:34:31.014631 1198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4769f6d36004a5ebed50495c422e951" candidate_uuid: "4c120b5093954ccfab5b89664a0e05db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bbf88280034cf6a5830591ce141f17" is_pre_election: true
I20260808 10:34:31.014763 1198 raft_consensus.cc:2454] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c120b5093954ccfab5b89664a0e05db in term 0.
I20260808 10:34:31.014763 896 raft_consensus.cc:2454] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c120b5093954ccfab5b89664a0e05db in term 0.
I20260808 10:34:31.015085 975 leader_election.cc:302] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c120b5093954ccfab5b89664a0e05db, f8bbf88280034cf6a5830591ce141f17; no voters:
I20260808 10:34:31.015312 1367 raft_consensus.cc:2781] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:31.015395 1367 raft_consensus.cc:482] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:31.015435 1367 raft_consensus.cc:3037] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:31.016721 1367 raft_consensus.cc:504] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:31.017100 1367 leader_election.cc:288] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [CANDIDATE]: Term 1 election: Requested vote from peers a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031), f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117)
I20260808 10:34:31.017488 1198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4769f6d36004a5ebed50495c422e951" candidate_uuid: "4c120b5093954ccfab5b89664a0e05db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bbf88280034cf6a5830591ce141f17"
I20260808 10:34:31.017613 1198 raft_consensus.cc:3037] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:31.017696 896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4769f6d36004a5ebed50495c422e951" candidate_uuid: "4c120b5093954ccfab5b89664a0e05db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3fa61780cab4954aaa4f257de1bf5be"
I20260808 10:34:31.017825 896 raft_consensus.cc:3037] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:31.018620 1198 raft_consensus.cc:2454] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c120b5093954ccfab5b89664a0e05db in term 1.
I20260808 10:34:31.018918 975 leader_election.cc:302] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c120b5093954ccfab5b89664a0e05db, f8bbf88280034cf6a5830591ce141f17; no voters:
I20260808 10:34:31.019088 1367 raft_consensus.cc:2781] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:31.019119 896 raft_consensus.cc:2454] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c120b5093954ccfab5b89664a0e05db in term 1.
I20260808 10:34:31.019187 1367 raft_consensus.cc:686] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 1 LEADER]: Becoming Leader. State: Replica: 4c120b5093954ccfab5b89664a0e05db, State: Running, Role: LEADER
I20260808 10:34:31.019320 1367 consensus_queue.cc:237] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } }
I20260808 10:34:31.021113 725 catalog_manager.cc:5495] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c120b5093954ccfab5b89664a0e05db (127.30.146.130). New cstate: current_term: 1 leader_uuid: "4c120b5093954ccfab5b89664a0e05db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c120b5093954ccfab5b89664a0e05db" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:31.040630 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:31.036867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42642 (local address 127.30.146.130:41841)
0808 10:34:31.037289 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:31.037311 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:31.037329 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:34:31.037379 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:31.037382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:31.037440 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:31.037531 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:31.037804 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:31.038410 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:31.039362 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:31.039613 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:31.039786 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:31.040090 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:31.040175 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:31.040262 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:31.040408 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 10:34:31.040462 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":150,"threads_started":1}
I20260808 10:34:31.048189 1198 raft_consensus.cc:1261] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Refusing update from remote peer 4c120b5093954ccfab5b89664a0e05db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:31.048401 896 raft_consensus.cc:1261] T a4769f6d36004a5ebed50495c422e951 P a3fa61780cab4954aaa4f257de1bf5be [term 1 FOLLOWER]: Refusing update from remote peer 4c120b5093954ccfab5b89664a0e05db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:31.048676 1367 consensus_queue.cc:1035] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8bbf88280034cf6a5830591ce141f17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:31.048839 1352 consensus_queue.cc:1035] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3fa61780cab4954aaa4f257de1bf5be" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:34:31.064707 1358 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 10:34:32.623638 723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:32.623799 723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:32.626461 723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e9315f5ee484844b5e34f76f3ef0d8f: Sending DeleteTablet for 3 replicas of tablet a4769f6d36004a5ebed50495c422e951
I20260808 10:34:32.627136 1025 tablet_service.cc:1514] Processing DeleteTablet for tablet a4769f6d36004a5ebed50495c422e951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:32 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:42634
I20260808 10:34:32.627295 873 tablet_service.cc:1514] Processing DeleteTablet for tablet a4769f6d36004a5ebed50495c422e951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:32 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:46322
I20260808 10:34:32.627547 1437 tablet_replica.cc:330] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db: stopping tablet replica
I20260808 10:34:32.627693 1437 raft_consensus.cc:2229] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:34:32.627687 1177 tablet_service.cc:1514] Processing DeleteTablet for tablet a4769f6d36004a5ebed50495c422e951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:32 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:55214
I20260808 10:34:32.627756 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 803
I20260808 10:34:32.630106 1437 raft_consensus.cc:2258] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:32.630944 1437 ts_tablet_manager.cc:1905] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:32.631887 1439 tablet_replica.cc:330] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17: stopping tablet replica
I20260808 10:34:32.632100 1439 raft_consensus.cc:2229] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:32.632272 1439 raft_consensus.cc:2258] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:32.633010 1439 ts_tablet_manager.cc:1905] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:32.634680 1437 ts_tablet_manager.cc:1918] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:34:32.634776 1437 log.cc:1196] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/a4769f6d36004a5ebed50495c422e951
I20260808 10:34:32.635099 1437 ts_tablet_manager.cc:1939] T a4769f6d36004a5ebed50495c422e951 P 4c120b5093954ccfab5b89664a0e05db: Deleting consensus metadata
I20260808 10:34:32.635787 1439 ts_tablet_manager.cc:1918] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:34:32.635921 1439 log.cc:1196] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/a4769f6d36004a5ebed50495c422e951
I20260808 10:34:32.636130 709 catalog_manager.cc:4841] TS 4c120b5093954ccfab5b89664a0e05db (127.30.146.130:41841): tablet a4769f6d36004a5ebed50495c422e951 (table test-table [id=0cb84472265c4373a8ceb5841d9cbcec]) successfully deleted
I20260808 10:34:32.636334 1439 ts_tablet_manager.cc:1939] T a4769f6d36004a5ebed50495c422e951 P f8bbf88280034cf6a5830591ce141f17: Deleting consensus metadata
I20260808 10:34:32.637313 709 catalog_manager.cc:4841] TS f8bbf88280034cf6a5830591ce141f17 (127.30.146.131:37117): tablet a4769f6d36004a5ebed50495c422e951 (table test-table [id=0cb84472265c4373a8ceb5841d9cbcec]) successfully deleted
W20260808 10:34:32.640475 711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:34031 (error 108)
I20260808 10:34:32.640745 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 957
I20260808 10:34:32.643263 1441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:32.642835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34031 (local address 127.0.0.1:33236)
0808 10:34:32.643099 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:32.643171 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":108,"threads_started":1}
W20260808 10:34:32.643410 711 catalog_manager.cc:4584] TS a3fa61780cab4954aaa4f257de1bf5be (127.30.146.129:34031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4769f6d36004a5ebed50495c422e951: Network error: Client connection negotiation failed: client connection to 127.30.146.129:34031: connect: Connection refused (error 111)
I20260808 10:34:32.650481 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 1105
I20260808 10:34:32.660451 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 691
2026-08-08T10:34:32Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6910 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:32 dist-test-slave-9gcw krb5kdc[1445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:32 dist-test-slave-9gcw krb5kdc[1445](info): set up 2 sockets
Aug 08 10:34:32 dist-test-slave-9gcw krb5kdc[1445](info): commencing operation
krb5kdc: starting...
W20260808 10:34:33.861042 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260808 10:34:34.034230 31306 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.173s user 0.002s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260808 10:34:34.159042 31306 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.125s user 0.001s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260808 10:34:34.336913 31306 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.178s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/test-user.keytab.
W20260808 10:34:34.932264 31306 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.595s user 0.002s sys 0.000s
Aug 08 10:34:34 dist-test-slave-9gcw krb5kdc[1445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
W20260808 10:34:35.317715 31306 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.30.146.190: real 0.358s user 0.000s sys 0.004s
I20260808 10:34:35.318459 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:46173
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:46173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:35.436874 1460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:35.437247 1460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:35.437358 1460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:35.441481 1460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:35.441586 1460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:35.441620 1460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:35.441661 1460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:35.441694 1460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:35.447250 1460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:46173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:46173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:35.448401 1460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:35.449494 1460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:35.458679 1466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:35.458807 1467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:35.458727 1469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:35.458943 1460 server_base.cc:1034] running on GCE node
I20260808 10:34:35.459527 1460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:35.460755 1460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:35.461956 1460 hybrid_clock.cc:648] HybridClock initialized: now 1786185275461930 us; error 38 us; skew 500 ppm
Aug 08 10:34:35 dist-test-slave-9gcw krb5kdc[1445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185275, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:35.464912 1460 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:34:35.465878 1460 webserver.cc:460] Webserver started at http://127.30.146.190:39939/ using document root <none> and password file <none>
I20260808 10:34:35.466450 1460 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:35.466512 1460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:35.466720 1460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:35.468904 1460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "ecc435f0534a45aab2b061646a7c02a4"
format_stamp: "Formatted at 2026-08-08 10:34:35 on dist-test-slave-9gcw"
server_key: "b6a459c087553ba16b508b31ee02591e"
server_key_iv: "e5980f6ef6689f7c27b9bc15633be826"
server_key_version: "encryptionkey@0"
I20260808 10:34:35.469633 1460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "ecc435f0534a45aab2b061646a7c02a4"
format_stamp: "Formatted at 2026-08-08 10:34:35 on dist-test-slave-9gcw"
server_key: "b6a459c087553ba16b508b31ee02591e"
server_key_iv: "e5980f6ef6689f7c27b9bc15633be826"
server_key_version: "encryptionkey@0"
I20260808 10:34:35.475039 1460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 10:34:35.478107 1476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:35.479779 1460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:34:35.479974 1460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "ecc435f0534a45aab2b061646a7c02a4"
format_stamp: "Formatted at 2026-08-08 10:34:35 on dist-test-slave-9gcw"
server_key: "b6a459c087553ba16b508b31ee02591e"
server_key_iv: "e5980f6ef6689f7c27b9bc15633be826"
server_key_version: "encryptionkey@0"
I20260808 10:34:35.480122 1460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:35.495513 1460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:35.499217 1460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:35.499457 1460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:35.507939 1460 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:46173
I20260808 10:34:35.507967 1532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:46173 every 8 connection(s)
I20260808 10:34:35.508996 1460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:35.511950 1533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:35.516683 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 1460
I20260808 10:34:35.516789 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:35.517107 31306 external_mini_cluster.cc:1428] Setting key 9c8e73eaad7f118b417aa11bc4287334
I20260808 10:34:35.517980 1533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4: Bootstrap starting.
I20260808 10:34:35.520457 1533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:35.521314 1533 log.cc:824] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:35.523288 1533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4: No bootstrap required, opened a new log
Aug 08 10:34:35 dist-test-slave-9gcw krb5kdc[1445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:35.527088 1533 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc435f0534a45aab2b061646a7c02a4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46173 } }
I20260808 10:34:35.527393 1533 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:35.527467 1533 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecc435f0534a45aab2b061646a7c02a4, State: Initialized, Role: FOLLOWER
I20260808 10:34:35.527995 1533 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc435f0534a45aab2b061646a7c02a4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46173 } }
I20260808 10:34:35.528173 1533 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:35.528249 1533 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:35.528348 1533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:35.529381 1533 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc435f0534a45aab2b061646a7c02a4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46173 } }
I20260808 10:34:35.529749 1533 leader_election.cc:302] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecc435f0534a45aab2b061646a7c02a4; no voters:
I20260808 10:34:35.530136 1533 leader_election.cc:288] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:35.530262 1538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:35.530519 1538 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [term 1 LEADER]: Becoming Leader. State: Replica: ecc435f0534a45aab2b061646a7c02a4, State: Running, Role: LEADER
I20260808 10:34:35.530898 1538 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc435f0534a45aab2b061646a7c02a4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46173 } }
I20260808 10:34:35.531533 1533 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:35.532897 1539 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecc435f0534a45aab2b061646a7c02a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc435f0534a45aab2b061646a7c02a4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46173 } } }
I20260808 10:34:35.532989 1536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:35.518708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.30.146.190:46173)
0808 10:34:35.519177 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:35.519193 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:35.519229 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:34:35.519894 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:35.519911 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:35.520240 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:35.520594 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:35.521490 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:35.522490 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:35.523391 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:35.523894 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:35.524086 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:35.526695 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:35.526700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:35.526714 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:35.526741 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:35.528973 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:35.529498 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:35.529505 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:35.529512 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:35.529654 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:35.530017 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:35.530023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:35.530026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:35.530607 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:35.531414 (+ 807us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:35.531569 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:34:35.531908 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":147,"threads_started":1}
I20260808 10:34:35.532898 1540 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecc435f0534a45aab2b061646a7c02a4. Latest consensus state: current_term: 1 leader_uuid: "ecc435f0534a45aab2b061646a7c02a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc435f0534a45aab2b061646a7c02a4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46173 } } }
I20260808 10:34:35.533113 1539 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:35.533111 1540 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4 [sys.catalog]: This master's current role is: LEADER
W20260808 10:34:35.536545 1553 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:34:35.536645 1553 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:34:35.536718 1551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:35.537617 1551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:35.543256 1551 catalog_manager.cc:1347] Generated new cluster ID: 24784da9d6354bdc98b33bc18a01e538
I20260808 10:34:35.543326 1551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:35.552824 1551 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:35.554306 1551 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:35.566957 1551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecc435f0534a45aab2b061646a7c02a4: Generated new TSK 0
I20260808 10:34:35.567935 1551 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:34:35.613199 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46173
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:35.745442 1560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:35.745760 1560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:35.745811 1560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:35.750002 1560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:35.750080 1560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:35.750142 1560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:35.756026 1560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:35.757237 1560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:35.758363 1560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:35.769554 1569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:35.769930 1567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:35.769771 1560 server_base.cc:1034] running on GCE node
W20260808 10:34:35.769554 1566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:35.770553 1560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:35.771322 1560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:35.772539 1560 hybrid_clock.cc:648] HybridClock initialized: now 1786185275772505 us; error 49 us; skew 500 ppm
Aug 08 10:34:35 dist-test-slave-9gcw krb5kdc[1445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185275, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:35.775769 1560 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:34:35.776755 1560 webserver.cc:460] Webserver started at http://127.30.146.129:42797/ using document root <none> and password file <none>
I20260808 10:34:35.777415 1560 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:35.777487 1560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:35.777665 1560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:35.779855 1560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "7793efb86fa24efaa74b8199521bd217"
format_stamp: "Formatted at 2026-08-08 10:34:35 on dist-test-slave-9gcw"
server_key: "e12176f210f8980062fb58801b0aab27"
server_key_iv: "cd2d77a3d4995c08214a62ad666603bf"
server_key_version: "encryptionkey@0"
I20260808 10:34:35.780444 1560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "7793efb86fa24efaa74b8199521bd217"
format_stamp: "Formatted at 2026-08-08 10:34:35 on dist-test-slave-9gcw"
server_key: "e12176f210f8980062fb58801b0aab27"
server_key_iv: "cd2d77a3d4995c08214a62ad666603bf"
server_key_version: "encryptionkey@0"
I20260808 10:34:35.785028 1560 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 10:34:35.788190 1578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:35.789793 1560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:35.789951 1560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "7793efb86fa24efaa74b8199521bd217"
format_stamp: "Formatted at 2026-08-08 10:34:35 on dist-test-slave-9gcw"
server_key: "e12176f210f8980062fb58801b0aab27"
server_key_iv: "cd2d77a3d4995c08214a62ad666603bf"
server_key_version: "encryptionkey@0"
I20260808 10:34:35.790084 1560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:35.809376 1560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:35.813124 1560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:35.813408 1560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:35.814117 1560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:35.815078 1560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:35.815131 1560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:35.815212 1560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:35.815246 1560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:35.825708 1560 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:38157
I20260808 10:34:35.825731 1700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:38157 every 8 connection(s)
I20260808 10:34:35.826817 1560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:35.832721 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 1560
I20260808 10:34:35.832890 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:35.833369 31306 external_mini_cluster.cc:1428] Setting key cb0b5cd83ad2b22a48d172aa3120810d
Aug 08 10:34:35 dist-test-slave-9gcw krb5kdc[1445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185275, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:35.842969 1536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:35.828780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:43961 (local address 127.30.146.190:46173)
0808 10:34:35.828945 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:35.828949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:35.829864 (+ 915us) server_negotiation.cc:407] Connection header received
0808 10:34:35.831788 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:35.831792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:35.831875 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:35.832007 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:35.833570 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:35.834235 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:35.835449 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:35.835706 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:35.835835 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:35.838943 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:35.838952 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:35.838955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:35.838981 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:35.840533 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:35.841133 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:35.841136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:35.841179 (+ 43us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:35.841278 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:35.841682 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:35.841686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:35.841688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:35.841913 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:35.842035 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:35.842607 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 10:34:35.842756 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:34:35.843894 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:35.829184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46173 (local address 127.30.146.129:43961)
0808 10:34:35.829687 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:35.829740 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:35.831534 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:35.832229 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:35.832237 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:35.832667 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:35.833401 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:35.833441 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:35.834383 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:35.834386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:35.835283 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:35.835293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:35.835883 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:35.835887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:35.836084 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:35.836750 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:35.836767 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:35.838747 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:35.840684 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:35.840691 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:35.840708 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:35.841005 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:35.841391 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:35.841395 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:35.841397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:35.841564 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:35.842036 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:35.842044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:35.842336 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 10:34:35.842544 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 10:34:35.842919 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:34:35.845445 1701 heartbeater.cc:347] Connected to a master server at 127.30.146.190:46173
I20260808 10:34:35.845731 1701 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:35.846300 1701 heartbeater.cc:511] Master 127.30.146.190:46173 requested a full tablet report, sending...
I20260808 10:34:35.848097 1495 ts_manager.cc:194] Registered new tserver with Master: 7793efb86fa24efaa74b8199521bd217 (127.30.146.129:38157)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:34:35.849711 1495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:43961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:34:35.869215 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46173
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:36.013922 1706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:36.014225 1706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:36.014269 1706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:36.018532 1706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:36.018600 1706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:36.018702 1706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:36.024070 1706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:36.025461 1706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:36.026557 1706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:36.038110 1711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:36.038210 1706 server_base.cc:1034] running on GCE node
W20260808 10:34:36.038110 1714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:36.038126 1712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:36.038995 1706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:36.039800 1706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:36.041045 1706 hybrid_clock.cc:648] HybridClock initialized: now 1786185276041021 us; error 37 us; skew 500 ppm
Aug 08 10:34:36 dist-test-slave-9gcw krb5kdc[1445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185276, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:36.044275 1706 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:34:36.045436 1706 webserver.cc:460] Webserver started at http://127.30.146.130:44221/ using document root <none> and password file <none>
I20260808 10:34:36.046162 1706 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:36.046279 1706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:36.046555 1706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:36.048969 1706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "5ec68f9dd25d4d43aee70ea0a2d6b8dc"
format_stamp: "Formatted at 2026-08-08 10:34:36 on dist-test-slave-9gcw"
server_key: "655abbccf357e08266cda38a65a163aa"
server_key_iv: "0311bd5cca1c4f0c5a540675f47f9912"
server_key_version: "encryptionkey@0"
I20260808 10:34:36.049719 1706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "5ec68f9dd25d4d43aee70ea0a2d6b8dc"
format_stamp: "Formatted at 2026-08-08 10:34:36 on dist-test-slave-9gcw"
server_key: "655abbccf357e08266cda38a65a163aa"
server_key_iv: "0311bd5cca1c4f0c5a540675f47f9912"
server_key_version: "encryptionkey@0"
I20260808 10:34:36.054248 1706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 10:34:36.057087 1721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:36.058569 1706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:34:36.058735 1706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "5ec68f9dd25d4d43aee70ea0a2d6b8dc"
format_stamp: "Formatted at 2026-08-08 10:34:36 on dist-test-slave-9gcw"
server_key: "655abbccf357e08266cda38a65a163aa"
server_key_iv: "0311bd5cca1c4f0c5a540675f47f9912"
server_key_version: "encryptionkey@0"
I20260808 10:34:36.058874 1706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:36.075553 1706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:36.078806 1706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:36.079021 1706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:36.079669 1706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:36.080541 1706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:36.080643 1706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:36.080737 1706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:36.080793 1706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:36.092272 1706 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:34491
I20260808 10:34:36.092284 1846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:34491 every 8 connection(s)
I20260808 10:34:36.093463 1706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:36.097604 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 1706
I20260808 10:34:36.097842 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:36.098189 31306 external_mini_cluster.cc:1428] Setting key 4f7091e6d97dcaa84ce789a04f8b4980
Aug 08 10:34:36 dist-test-slave-9gcw krb5kdc[1445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185276, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:36.110059 1536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:36.095580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:42631 (local address 127.30.146.190:46173)
0808 10:34:36.095729 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:36.095733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:36.096625 (+ 892us) server_negotiation.cc:407] Connection header received
0808 10:34:36.098223 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:36.098226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:36.098290 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:36.098414 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:36.099817 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:36.100514 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:36.101924 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:36.102172 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:36.102305 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:36.105701 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:36.105704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:36.105706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:36.105732 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:36.107288 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:36.108107 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:36.108110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:36.108111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:36.108187 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:36.108669 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:36.108671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:36.108672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:36.108859 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:36.108933 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:36.109802 (+ 869us) server_negotiation.cc:299] Negotiation successful
0808 10:34:36.109927 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:34:36.111711 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:36.095993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46173 (local address 127.30.146.130:42631)
0808 10:34:36.096455 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:36.096498 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:36.097956 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:36.098605 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:36.098614 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:36.099040 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:36.099606 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:36.099636 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:36.100676 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:36.100679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:36.101746 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:36.101756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:36.102367 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:36.102372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:36.102714 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:36.103558 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:36.103575 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:36.105500 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:36.107450 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:36.107462 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:36.107484 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:36.107974 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:36.108298 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:36.108304 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:36.108310 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:36.108548 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:36.108978 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:36.108987 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:36.109472 (+ 485us) client_negotiation.cc:769] Sending connection context
0808 10:34:36.109741 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 10:34:36.110209 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:34:36.113368 1847 heartbeater.cc:347] Connected to a master server at 127.30.146.190:46173
I20260808 10:34:36.113646 1847 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:36.114248 1847 heartbeater.cc:511] Master 127.30.146.190:46173 requested a full tablet report, sending...
I20260808 10:34:36.115455 1495 ts_manager.cc:194] Registered new tserver with Master: 5ec68f9dd25d4d43aee70ea0a2d6b8dc (127.30.146.130:34491)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
I20260808 10:34:36.116437 1495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:42631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:34:36.137948 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46173
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:36.264727 1852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:36.265029 1852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:36.265074 1852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:36.269309 1852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:36.269378 1852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:36.269474 1852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:36.274956 1852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:36.276144 1852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:36.277312 1852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:36.288604 1860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:36.288800 1857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:36.288651 1858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:36.289021 1852 server_base.cc:1034] running on GCE node
I20260808 10:34:36.289552 1852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:36.290308 1852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:36.291536 1852 hybrid_clock.cc:648] HybridClock initialized: now 1786185276291514 us; error 35 us; skew 500 ppm
Aug 08 10:34:36 dist-test-slave-9gcw krb5kdc[1445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185276, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:36.294718 1852 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:34:36.295766 1852 webserver.cc:460] Webserver started at http://127.30.146.131:32983/ using document root <none> and password file <none>
I20260808 10:34:36.296386 1852 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:36.296489 1852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:36.296729 1852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:36.299422 1852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "684ae463a7ed4487908ae197214e23ee"
format_stamp: "Formatted at 2026-08-08 10:34:36 on dist-test-slave-9gcw"
server_key: "7d6eb4ae86ff7433a52026e2bdbe9266"
server_key_iv: "d45baafec61dad8632fe577a3caa9e25"
server_key_version: "encryptionkey@0"
I20260808 10:34:36.300038 1852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "684ae463a7ed4487908ae197214e23ee"
format_stamp: "Formatted at 2026-08-08 10:34:36 on dist-test-slave-9gcw"
server_key: "7d6eb4ae86ff7433a52026e2bdbe9266"
server_key_iv: "d45baafec61dad8632fe577a3caa9e25"
server_key_version: "encryptionkey@0"
I20260808 10:34:36.304389 1852 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 10:34:36.307145 1867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:36.308537 1852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:34:36.308686 1852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "684ae463a7ed4487908ae197214e23ee"
format_stamp: "Formatted at 2026-08-08 10:34:36 on dist-test-slave-9gcw"
server_key: "7d6eb4ae86ff7433a52026e2bdbe9266"
server_key_iv: "d45baafec61dad8632fe577a3caa9e25"
server_key_version: "encryptionkey@0"
I20260808 10:34:36.308818 1852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:36.325557 1852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:36.329255 1852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:36.329486 1852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:36.330147 1852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:36.331044 1852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:36.331120 1852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:36.331218 1852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:36.331264 1852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:36.340966 1852 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:43697
I20260808 10:34:36.341005 1991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:43697 every 8 connection(s)
I20260808 10:34:36.342240 1852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:36.346890 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 1852
I20260808 10:34:36.347018 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:36.347352 31306 external_mini_cluster.cc:1428] Setting key 57449e84acd55e198f0a0cc89794b84c
Aug 08 10:34:36 dist-test-slave-9gcw krb5kdc[1445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185276, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:36.358304 1536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:36.344533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:43325 (local address 127.30.146.190:46173)
0808 10:34:36.344697 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:36.344701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:36.345877 (+ 1176us) server_negotiation.cc:407] Connection header received
0808 10:34:36.347422 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:36.347425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:36.347490 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:36.347596 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:36.349136 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:36.349726 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:36.350927 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:36.351159 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:36.351287 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:36.354333 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:36.354337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:36.354339 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:36.354367 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:36.356052 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:36.356605 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:36.356608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:36.356610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:36.356718 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:36.357058 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:36.357060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:36.357061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:36.357368 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:36.357452 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:36.357994 (+ 542us) server_negotiation.cc:299] Negotiation successful
0808 10:34:36.358138 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:34:36.359153 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:36.345098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46173 (local address 127.30.146.131:43325)
0808 10:34:36.345680 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:36.345729 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:36.347179 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:36.347831 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:36.347842 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:36.348305 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:36.348960 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:36.348986 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:36.349877 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:36.349881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:36.350775 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:36.350786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:36.351336 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:36.351341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:36.351530 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:36.352203 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:36.352220 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:36.354127 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:36.356195 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:36.356200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:36.356215 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:36.356496 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:36.356819 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:36.356821 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:36.356822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:36.356967 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:36.357463 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:36.357469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:36.357730 (+ 261us) client_negotiation.cc:769] Sending connection context
0808 10:34:36.357924 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 10:34:36.358231 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":137,"threads_started":1}
I20260808 10:34:36.360476 1992 heartbeater.cc:347] Connected to a master server at 127.30.146.190:46173
I20260808 10:34:36.360785 1992 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:36.361545 1992 heartbeater.cc:511] Master 127.30.146.190:46173 requested a full tablet report, sending...
I20260808 10:34:36.362664 1495 ts_manager.cc:194] Registered new tserver with Master: 684ae463a7ed4487908ae197214e23ee (127.30.146.131:43697)
I20260808 10:34:36.363343 1495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:43325
I20260808 10:34:36.374327 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:34:36.390196 1536 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:34:36.388731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46230 (local address 127.30.146.190:46173)
0808 10:34:36.388873 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:36.388877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:36.389022 (+ 145us) server_negotiation.cc:407] Connection header received
0808 10:34:36.389186 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:36.389193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:36.389242 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:36.389331 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:36.390072 (+ 741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46230: BlockingRecv error: recv got EOF from 127.0.0.1:46230 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:34:36.391585 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 1560
I20260808 10:34:36.403338 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 1706
I20260808 10:34:36.412635 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 1852
I20260808 10:34:36.421886 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 1460
2026-08-08T10:34:36Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3757 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:36 dist-test-slave-9gcw krb5kdc[2004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:36 dist-test-slave-9gcw krb5kdc[2004](info): set up 2 sockets
Aug 08 10:34:36 dist-test-slave-9gcw krb5kdc[2004](info): commencing operation
krb5kdc: starting...
W20260808 10:34:37.626729 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:37 dist-test-slave-9gcw krb5kdc[2004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:34:37.769358 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:41957
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:41957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:37.891991 2017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:37.892344 2017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:37.892422 2017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:37.896814 2017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:37.896930 2017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:37.896976 2017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:37.897006 2017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:37.897042 2017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:37.902688 2017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:41957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:41957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:37.903908 2017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:37.905035 2017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:37.913879 2023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:37.913936 2022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:37.913939 2025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:37.914556 2017 server_base.cc:1034] running on GCE node
I20260808 10:34:37.915161 2017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:37.916369 2017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:37.917588 2017 hybrid_clock.cc:648] HybridClock initialized: now 1786185277917575 us; error 47 us; skew 500 ppm
Aug 08 10:34:37 dist-test-slave-9gcw krb5kdc[2004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185277, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:37.935688 2017 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:34:37.936939 2017 webserver.cc:460] Webserver started at http://127.30.146.190:43215/ using document root <none> and password file <none>
I20260808 10:34:37.937696 2017 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:37.937819 2017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:37.938099 2017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:37.940600 2017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "3c559fa641af4403b44892d2ab992d2f"
format_stamp: "Formatted at 2026-08-08 10:34:37 on dist-test-slave-9gcw"
server_key: "fbe6d471135c4d35d6b7067ecbd98fdd"
server_key_iv: "2a90a0d201cddd3d3db968b0126ab56d"
server_key_version: "encryptionkey@0"
I20260808 10:34:37.941424 2017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "3c559fa641af4403b44892d2ab992d2f"
format_stamp: "Formatted at 2026-08-08 10:34:37 on dist-test-slave-9gcw"
server_key: "fbe6d471135c4d35d6b7067ecbd98fdd"
server_key_iv: "2a90a0d201cddd3d3db968b0126ab56d"
server_key_version: "encryptionkey@0"
I20260808 10:34:37.945875 2017 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:34:37.948844 2034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:37.950268 2017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:37.950453 2017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "3c559fa641af4403b44892d2ab992d2f"
format_stamp: "Formatted at 2026-08-08 10:34:37 on dist-test-slave-9gcw"
server_key: "fbe6d471135c4d35d6b7067ecbd98fdd"
server_key_iv: "2a90a0d201cddd3d3db968b0126ab56d"
server_key_version: "encryptionkey@0"
I20260808 10:34:37.950598 2017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:37.978520 2017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:37.981985 2017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:37.982228 2017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:37.990217 2017 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:41957
I20260808 10:34:37.990549 2089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:41957 every 8 connection(s)
I20260808 10:34:37.991305 2017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:37.994305 2090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:37.998226 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 2017
I20260808 10:34:37.998333 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:37.998644 31306 external_mini_cluster.cc:1428] Setting key d1ccfe5b3976671ffc9d2c54e1f3a5f7
I20260808 10:34:38.000514 2090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f: Bootstrap starting.
I20260808 10:34:38.003201 2090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:38.003985 2090 log.cc:824] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:38.006441 2090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f: No bootstrap required, opened a new log
Aug 08 10:34:38 dist-test-slave-9gcw krb5kdc[2004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:38.011070 2090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c559fa641af4403b44892d2ab992d2f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41957 } }
I20260808 10:34:38.011381 2090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:38.011461 2090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c559fa641af4403b44892d2ab992d2f, State: Initialized, Role: FOLLOWER
I20260808 10:34:38.012014 2090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c559fa641af4403b44892d2ab992d2f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41957 } }
I20260808 10:34:38.012197 2090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:38.012322 2090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:38.012456 2090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:38.013978 2090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c559fa641af4403b44892d2ab992d2f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41957 } }
I20260808 10:34:38.014353 2090 leader_election.cc:302] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c559fa641af4403b44892d2ab992d2f; no voters:
I20260808 10:34:38.014735 2090 leader_election.cc:288] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:38.014894 2095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:38.015137 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:38.000247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38706 (local address 127.30.146.190:41957)
0808 10:34:38.000735 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:38.000753 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:38.000791 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:34:38.001575 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:38.001596 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:38.001943 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:38.002239 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:38.003292 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:38.004554 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:38.005724 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:38.006460 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:38.006711 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:38.009321 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:38.009331 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:38.009356 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:38.009400 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:38.011517 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:38.011987 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:38.011992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:38.011998 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:38.012190 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:38.012546 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:38.012550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:38.012552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:38.013138 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:38.013379 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:38.013652 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 10:34:38.014037 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":318,"thread_start_us":151,"threads_started":1}
I20260808 10:34:38.015146 2095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [term 1 LEADER]: Becoming Leader. State: Replica: 3c559fa641af4403b44892d2ab992d2f, State: Running, Role: LEADER
I20260808 10:34:38.015764 2095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c559fa641af4403b44892d2ab992d2f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41957 } }
I20260808 10:34:38.016139 2090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:38.017675 2097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c559fa641af4403b44892d2ab992d2f. Latest consensus state: current_term: 1 leader_uuid: "3c559fa641af4403b44892d2ab992d2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c559fa641af4403b44892d2ab992d2f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41957 } } }
I20260808 10:34:38.017688 2096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c559fa641af4403b44892d2ab992d2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c559fa641af4403b44892d2ab992d2f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41957 } } }
I20260808 10:34:38.017841 2096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:38.017841 2097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:38.018285 2104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:38.019045 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:34:38.021301 2104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:38.028055 2104 catalog_manager.cc:1347] Generated new cluster ID: c85aaf2e168b453dbeae11574054a14f
I20260808 10:34:38.028126 2104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:38.039333 2104 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:38.040717 2104 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:38.045459 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:38.039156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.30.146.190:41957)
0808 10:34:38.039326 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:38.039330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:38.039682 (+ 352us) server_negotiation.cc:407] Connection header received
0808 10:34:38.039814 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:38.039817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:38.039869 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:38.039962 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:38.040630 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:38.041520 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:38.042734 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:38.043105 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:38.043215 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:38.044261 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:38.044265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:38.044267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:38.044789 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:38.044839 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:38.044859 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:38.044958 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:38.045187 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 10:34:38.045279 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 10:34:38.045569 2104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c559fa641af4403b44892d2ab992d2f: Generated new TSK 0
I20260808 10:34:38.046658 2104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:34:38.057502 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 2017
2026-08-08T10:34:38Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1635 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:38 dist-test-slave-9gcw krb5kdc[2125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:38 dist-test-slave-9gcw krb5kdc[2125](info): set up 2 sockets
Aug 08 10:34:38 dist-test-slave-9gcw krb5kdc[2125](info): commencing operation
krb5kdc: starting...
W20260808 10:34:39.232980 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:39 dist-test-slave-9gcw krb5kdc[2125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:34:39.373430 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:41151
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:41151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:39.493454 2139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:39.493811 2139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:39.493889 2139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:39.498224 2139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:39.498322 2139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:39.498361 2139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:39.498400 2139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:39.498435 2139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:39.504117 2139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:41151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:41151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:39.505445 2139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:39.506531 2139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:39.515841 2148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:39.515859 2144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:39.515934 2139 server_base.cc:1034] running on GCE node
W20260808 10:34:39.515859 2145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:39.516680 2139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:39.517973 2139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:39.519182 2139 hybrid_clock.cc:648] HybridClock initialized: now 1786185279519153 us; error 41 us; skew 500 ppm
Aug 08 10:34:39 dist-test-slave-9gcw krb5kdc[2125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185279, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:39.522326 2139 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:34:39.523273 2139 webserver.cc:460] Webserver started at http://127.30.146.190:44335/ using document root <none> and password file <none>
I20260808 10:34:39.523897 2139 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:39.524003 2139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:39.524258 2139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:39.526533 2139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "3597f561210449c7b2c561ae7c1d2b1b"
format_stamp: "Formatted at 2026-08-08 10:34:39 on dist-test-slave-9gcw"
server_key: "5b732c440a658bfa2962bebaf8039044"
server_key_iv: "cdee5b59ed47031e905388c6e2803097"
server_key_version: "encryptionkey@0"
I20260808 10:34:39.527212 2139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "3597f561210449c7b2c561ae7c1d2b1b"
format_stamp: "Formatted at 2026-08-08 10:34:39 on dist-test-slave-9gcw"
server_key: "5b732c440a658bfa2962bebaf8039044"
server_key_iv: "cdee5b59ed47031e905388c6e2803097"
server_key_version: "encryptionkey@0"
I20260808 10:34:39.531502 2139 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:34:39.534523 2155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:39.535985 2139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:34:39.536193 2139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "3597f561210449c7b2c561ae7c1d2b1b"
format_stamp: "Formatted at 2026-08-08 10:34:39 on dist-test-slave-9gcw"
server_key: "5b732c440a658bfa2962bebaf8039044"
server_key_iv: "cdee5b59ed47031e905388c6e2803097"
server_key_version: "encryptionkey@0"
I20260808 10:34:39.536329 2139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:39.553386 2139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:39.557402 2139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:39.557644 2139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:39.565789 2139 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:41151
I20260808 10:34:39.566092 2214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:41151 every 8 connection(s)
I20260808 10:34:39.566942 2139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:39.570014 2215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:39.571748 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 2139
I20260808 10:34:39.571866 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:39.572206 31306 external_mini_cluster.cc:1428] Setting key 7159066e204fa1d003489490d229ba6e
I20260808 10:34:39.577325 2215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b: Bootstrap starting.
I20260808 10:34:39.580075 2215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:39.580925 2215 log.cc:824] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:34:39 dist-test-slave-9gcw krb5kdc[2125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:39.583106 2215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b: No bootstrap required, opened a new log
I20260808 10:34:39.588265 2215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f561210449c7b2c561ae7c1d2b1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41151 } }
I20260808 10:34:39.588580 2215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:39.588634 2215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3597f561210449c7b2c561ae7c1d2b1b, State: Initialized, Role: FOLLOWER
I20260808 10:34:39.589241 2218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:39.573782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34190 (local address 127.30.146.190:41151)
0808 10:34:39.574236 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:39.574276 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:39.574326 (+ 50us) server_negotiation.cc:407] Connection header received
0808 10:34:39.575226 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:39.575248 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:39.575707 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:39.576089 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:39.576932 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:39.578462 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:39.579405 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:39.580116 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:39.580303 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:39.583244 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:39.583250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:39.583267 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:39.583299 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:39.585540 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:39.586109 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:39.586115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:39.586121 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:39.586272 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:39.586615 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:39.586624 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:39.586626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:39.587215 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:39.587434 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:39.587779 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 10:34:39.588124 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":110,"threads_started":1}
I20260808 10:34:39.589174 2215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f561210449c7b2c561ae7c1d2b1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41151 } }
I20260808 10:34:39.589366 2215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:39.589428 2215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:39.589536 2215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:39.590768 2215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f561210449c7b2c561ae7c1d2b1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41151 } }
I20260808 10:34:39.591151 2215 leader_election.cc:302] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3597f561210449c7b2c561ae7c1d2b1b; no voters:
I20260808 10:34:39.591509 2215 leader_election.cc:288] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:39.591631 2220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:39.591881 2220 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [term 1 LEADER]: Becoming Leader. State: Replica: 3597f561210449c7b2c561ae7c1d2b1b, State: Running, Role: LEADER
I20260808 10:34:39.592293 2220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f561210449c7b2c561ae7c1d2b1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41151 } }
I20260808 10:34:39.592857 2215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:39.594048 2221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3597f561210449c7b2c561ae7c1d2b1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f561210449c7b2c561ae7c1d2b1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41151 } } }
I20260808 10:34:39.594205 2221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:39.594548 2223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3597f561210449c7b2c561ae7c1d2b1b. Latest consensus state: current_term: 1 leader_uuid: "3597f561210449c7b2c561ae7c1d2b1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f561210449c7b2c561ae7c1d2b1b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41151 } } }
I20260808 10:34:39.594727 2223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:39.594682 2230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:39.598169 2230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:39.604302 2230 catalog_manager.cc:1347] Generated new cluster ID: 9404ad06aa1148a0b69c02fe35fa62aa
I20260808 10:34:39.604388 2230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:39.626755 2230 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:39.627964 2230 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:39.642586 2230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3597f561210449c7b2c561ae7c1d2b1b: Generated new TSK 0
I20260808 10:34:39.643406 2230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:34:39.685097 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41151
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:39.807348 2243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:39.807654 2243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:39.807760 2243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:39.811957 2243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:39.812074 2243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:39.812243 2243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:39.817788 2243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:39.818899 2243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:39.819931 2243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:39.830529 2249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:39.830559 2248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:39.830529 2251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:39.830890 2243 server_base.cc:1034] running on GCE node
I20260808 10:34:39.831470 2243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:39.832149 2243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:39.833338 2243 hybrid_clock.cc:648] HybridClock initialized: now 1786185279833294 us; error 59 us; skew 500 ppm
Aug 08 10:34:39 dist-test-slave-9gcw krb5kdc[2125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185279, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:39.836637 2243 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:34:39.837689 2243 webserver.cc:460] Webserver started at http://127.30.146.129:44521/ using document root <none> and password file <none>
I20260808 10:34:39.838266 2243 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:39.838331 2243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:39.838508 2243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:39.840806 2243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "15ae523695ec461d9d9f60dd115ffb37"
format_stamp: "Formatted at 2026-08-08 10:34:39 on dist-test-slave-9gcw"
server_key: "2c0e36b1ce1c5a0714ee94bb223f59ca"
server_key_iv: "829bfc0c14d11f8db27e3afe9cc806c7"
server_key_version: "encryptionkey@0"
I20260808 10:34:39.841590 2243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "15ae523695ec461d9d9f60dd115ffb37"
format_stamp: "Formatted at 2026-08-08 10:34:39 on dist-test-slave-9gcw"
server_key: "2c0e36b1ce1c5a0714ee94bb223f59ca"
server_key_iv: "829bfc0c14d11f8db27e3afe9cc806c7"
server_key_version: "encryptionkey@0"
I20260808 10:34:39.845750 2243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:39.848588 2258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:39.850075 2243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:39.850207 2243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "15ae523695ec461d9d9f60dd115ffb37"
format_stamp: "Formatted at 2026-08-08 10:34:39 on dist-test-slave-9gcw"
server_key: "2c0e36b1ce1c5a0714ee94bb223f59ca"
server_key_iv: "829bfc0c14d11f8db27e3afe9cc806c7"
server_key_version: "encryptionkey@0"
I20260808 10:34:39.850312 2243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:39.865974 2243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:39.869488 2243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:39.869663 2243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:39.870232 2243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:39.871042 2243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:39.871089 2243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:39.871140 2243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:39.871158 2243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:39.880577 2243 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:43359
I20260808 10:34:39.880610 2378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:43359 every 8 connection(s)
I20260808 10:34:39.881623 2243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:39.883668 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 2243
I20260808 10:34:39.883834 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:39.884145 31306 external_mini_cluster.cc:1428] Setting key 06241c9be436702d3ec4be91081573e0
Aug 08 10:34:39 dist-test-slave-9gcw krb5kdc[2125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185279, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:34:39.899384 2218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:39.883617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:36771 (local address 127.30.146.190:41151)
0808 10:34:39.883875 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:39.883881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:39.884793 (+ 912us) server_negotiation.cc:407] Connection header received
0808 10:34:39.886522 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:39.886526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:39.886576 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:39.886658 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:39.887943 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:39.888519 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:39.890415 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:39.890735 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:39.890834 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:39.894771 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:39.894775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:39.894780 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:39.894943 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:39.896766 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:39.897526 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:39.897529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:39.897530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:39.897615 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:39.898078 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:39.898081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:39.898082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:39.898308 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:39.898391 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:39.899058 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 10:34:39.899230 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:34:39.900383 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:39.884061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:41151 (local address 127.30.146.129:36771)
0808 10:34:39.884633 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:39.884681 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:39.886306 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:39.886842 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:39.886848 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:39.887222 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:39.887717 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:39.887731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:39.888669 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:39.888675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:39.890260 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:39.890276 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:39.890932 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:39.890939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:39.891189 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:39.891945 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:39.891973 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:39.894489 (+ 2516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:39.896943 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:39.896954 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:39.896984 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:39.897404 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:39.897761 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:39.897764 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:39.897765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:39.897960 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:39.898464 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:39.898470 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:39.898753 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 10:34:39.898933 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 10:34:39.899316 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":167,"threads_started":1}
I20260808 10:34:39.901957 2379 heartbeater.cc:347] Connected to a master server at 127.30.146.190:41151
I20260808 10:34:39.902257 2379 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:39.902923 2379 heartbeater.cc:511] Master 127.30.146.190:41151 requested a full tablet report, sending...
I20260808 10:34:39.904630 2173 ts_manager.cc:194] Registered new tserver with Master: 15ae523695ec461d9d9f60dd115ffb37 (127.30.146.129:43359)
I20260808 10:34:39.905926 2173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:36771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:34:39.917248 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41151
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:40.039000 2385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:40.039310 2385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:40.039359 2385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:40.044286 2385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:40.044364 2385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:40.044432 2385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:40.050271 2385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:40.051373 2385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:40.052398 2385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:40.062740 2390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:40.062729 2391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:40.062795 2385 server_base.cc:1034] running on GCE node
W20260808 10:34:40.062723 2394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:40.063367 2385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:40.064069 2385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:40.065238 2385 hybrid_clock.cc:648] HybridClock initialized: now 1786185280065208 us; error 50 us; skew 500 ppm
Aug 08 10:34:40 dist-test-slave-9gcw krb5kdc[2125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185280, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:40.068295 2385 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:34:40.069321 2385 webserver.cc:460] Webserver started at http://127.30.146.130:33527/ using document root <none> and password file <none>
I20260808 10:34:40.070231 2385 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:40.070330 2385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:40.070619 2385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:40.072700 2385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "284628f9946a4cbdb47b1584221ea54e"
format_stamp: "Formatted at 2026-08-08 10:34:40 on dist-test-slave-9gcw"
server_key: "69570538f2e9c2be4a157472ac8a04f6"
server_key_iv: "90f451a79cee0438e6408c300e0ec6b7"
server_key_version: "encryptionkey@0"
I20260808 10:34:40.073319 2385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "284628f9946a4cbdb47b1584221ea54e"
format_stamp: "Formatted at 2026-08-08 10:34:40 on dist-test-slave-9gcw"
server_key: "69570538f2e9c2be4a157472ac8a04f6"
server_key_iv: "90f451a79cee0438e6408c300e0ec6b7"
server_key_version: "encryptionkey@0"
I20260808 10:34:40.077553 2385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:34:40.080174 2402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:40.081743 2385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:40.081895 2385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "284628f9946a4cbdb47b1584221ea54e"
format_stamp: "Formatted at 2026-08-08 10:34:40 on dist-test-slave-9gcw"
server_key: "69570538f2e9c2be4a157472ac8a04f6"
server_key_iv: "90f451a79cee0438e6408c300e0ec6b7"
server_key_version: "encryptionkey@0"
I20260808 10:34:40.082026 2385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:40.117934 2385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:40.121371 2385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:40.121603 2385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:40.122265 2385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:40.123140 2385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:40.123217 2385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:40.123291 2385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:40.123343 2385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:40.132598 2385 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:37239
I20260808 10:34:40.132627 2525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:37239 every 8 connection(s)
I20260808 10:34:40.133548 2385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:40.135699 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 2385
I20260808 10:34:40.135833 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:40.136106 31306 external_mini_cluster.cc:1428] Setting key 437d2f12d8c3e894603f5e5886a02edc
Aug 08 10:34:40 dist-test-slave-9gcw krb5kdc[2125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185280, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:34:40.149464 2218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:40.135420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:41633 (local address 127.30.146.190:41151)
0808 10:34:40.135618 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:40.135623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:40.136664 (+ 1041us) server_negotiation.cc:407] Connection header received
0808 10:34:40.138463 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:40.138467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:40.138520 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:40.138704 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:40.140388 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:40.140996 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:40.142336 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:40.142583 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:40.142705 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:40.145689 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:40.145692 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:40.145694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:40.145718 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:40.147322 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:40.147864 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:40.147868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:40.147869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:40.147963 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:40.148318 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:40.148321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:40.148323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:40.148514 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:40.148638 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:40.149160 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 10:34:40.149285 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
I20260808 10:34:40.150314 2528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:40.135928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:41151 (local address 127.30.146.130:41633)
0808 10:34:40.136473 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:40.136542 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:40.138200 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:40.138919 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:40.138931 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:40.139462 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:40.140224 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:40.140245 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:40.141128 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:40.141131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:40.142197 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:40.142207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:40.142755 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:40.142760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:40.142941 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:40.143614 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:40.143638 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:40.145494 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:40.147458 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:40.147464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:40.147478 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:40.147751 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:40.148062 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:40.148066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:40.148068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:40.148211 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:40.148633 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:40.148640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:40.148891 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 10:34:40.149075 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 10:34:40.149423 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":149,"threads_started":1}
I20260808 10:34:40.151708 2526 heartbeater.cc:347] Connected to a master server at 127.30.146.190:41151
I20260808 10:34:40.151961 2526 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:40.152590 2526 heartbeater.cc:511] Master 127.30.146.190:41151 requested a full tablet report, sending...
I20260808 10:34:40.153888 2173 ts_manager.cc:194] Registered new tserver with Master: 284628f9946a4cbdb47b1584221ea54e (127.30.146.130:37239)
I20260808 10:34:40.154752 2173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:41633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:34:40.168836 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41151
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:40.284626 2531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:40.284998 2531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:40.285050 2531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:40.289361 2531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:40.289440 2531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:40.289502 2531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:40.295908 2531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:40.297026 2531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:40.298208 2531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:40.308427 2537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:40.308269 2540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:40.308336 2536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:40.308717 2531 server_base.cc:1034] running on GCE node
I20260808 10:34:40.309134 2531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:40.309821 2531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:40.311002 2531 hybrid_clock.cc:648] HybridClock initialized: now 1786185280310963 us; error 52 us; skew 500 ppm
Aug 08 10:34:40 dist-test-slave-9gcw krb5kdc[2125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185280, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:40.314222 2531 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:34:40.315152 2531 webserver.cc:460] Webserver started at http://127.30.146.131:36667/ using document root <none> and password file <none>
I20260808 10:34:40.315750 2531 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:40.315819 2531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:40.316043 2531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:40.318214 2531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "08354ce8a48b4240921139b003f5f958"
format_stamp: "Formatted at 2026-08-08 10:34:40 on dist-test-slave-9gcw"
server_key: "db9bca8909230364a52eaa7ee1f93759"
server_key_iv: "ee9801744aaaa5b71d0312349bf30e20"
server_key_version: "encryptionkey@0"
I20260808 10:34:40.318789 2531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "08354ce8a48b4240921139b003f5f958"
format_stamp: "Formatted at 2026-08-08 10:34:40 on dist-test-slave-9gcw"
server_key: "db9bca8909230364a52eaa7ee1f93759"
server_key_iv: "ee9801744aaaa5b71d0312349bf30e20"
server_key_version: "encryptionkey@0"
I20260808 10:34:40.322599 2531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:40.325245 2550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:40.326561 2531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:34:40.326700 2531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "08354ce8a48b4240921139b003f5f958"
format_stamp: "Formatted at 2026-08-08 10:34:40 on dist-test-slave-9gcw"
server_key: "db9bca8909230364a52eaa7ee1f93759"
server_key_iv: "ee9801744aaaa5b71d0312349bf30e20"
server_key_version: "encryptionkey@0"
I20260808 10:34:40.326830 2531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:40.341833 2531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:40.345207 2531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:40.345443 2531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:40.346069 2531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:40.346940 2531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:40.347018 2531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:40.347086 2531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:40.347121 2531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:40.357031 2531 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:41279
I20260808 10:34:40.357054 2672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:41279 every 8 connection(s)
I20260808 10:34:40.358103 2531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:40.366909 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 2531
I20260808 10:34:40.367080 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:40.367475 31306 external_mini_cluster.cc:1428] Setting key f1b1e0a32309294e8f048054cbd31d73
Aug 08 10:34:40 dist-test-slave-9gcw krb5kdc[2125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185280, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:40.372642 2218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:40.360127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:54451 (local address 127.30.146.190:41151)
0808 10:34:40.360310 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:40.360316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:40.361115 (+ 799us) server_negotiation.cc:407] Connection header received
0808 10:34:40.362517 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:40.362520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:40.362570 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:40.362647 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:40.363815 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:40.364456 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:40.365738 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:40.365957 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:40.366041 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:40.368880 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:40.368888 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:40.368891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:40.368916 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:40.370485 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:40.371027 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:40.371030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:40.371032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:40.371119 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:40.371477 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:40.371482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:40.371483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:40.371683 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:40.371795 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:40.372338 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 10:34:40.372468 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:34:40.373554 2675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:40.360478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:41151 (local address 127.30.146.131:54451)
0808 10:34:40.360958 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:40.361001 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:40.362314 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:40.362831 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:40.362837 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:40.363205 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:40.363673 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:40.363685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:40.364569 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:40.364572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:40.365635 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:40.365644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:40.366126 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:40.366130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:40.366284 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:40.366944 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:40.366971 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:40.368707 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:40.370618 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:40.370623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:40.370638 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:40.370913 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:40.371218 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:40.371221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:40.371223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:40.371375 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:40.371808 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:40.371817 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:40.372075 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 10:34:40.372279 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 10:34:40.372620 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":122,"threads_started":1}
I20260808 10:34:40.374754 2673 heartbeater.cc:347] Connected to a master server at 127.30.146.190:41151
I20260808 10:34:40.375031 2673 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:40.375581 2673 heartbeater.cc:511] Master 127.30.146.190:41151 requested a full tablet report, sending...
I20260808 10:34:40.376614 2173 ts_manager.cc:194] Registered new tserver with Master: 08354ce8a48b4240921139b003f5f958 (127.30.146.131:41279)
I20260808 10:34:40.377301 2173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:54451
I20260808 10:34:40.381731 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:34:40 dist-test-slave-9gcw krb5kdc[2125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185280, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 10:34:40 dist-test-slave-9gcw krb5kdc[2125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185280, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:40.410029 2218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:40.401118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34196 (local address 127.30.146.190:41151)
0808 10:34:40.401316 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:40.401321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:40.401432 (+ 111us) server_negotiation.cc:407] Connection header received
0808 10:34:40.401613 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:40.401617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:40.401672 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:40.401785 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:40.402530 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:40.403062 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:40.403923 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:40.404149 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:40.404271 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:40.406824 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:40.406828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:40.406830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:40.406857 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:40.408275 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:40.408696 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:40.408699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:40.408701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:40.408802 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:40.409054 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:40.409056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:40.409058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:40.409452 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:40.409589 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:40.409793 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 10:34:40.409900 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 10:34:40.410542 2173 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34196
I20260808 10:34:40.411669 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 2243
I20260808 10:34:40.421178 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 2385
I20260808 10:34:40.430095 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 2531
I20260808 10:34:40.437516 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 2139
2026-08-08T10:34:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2380 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:40 dist-test-slave-9gcw krb5kdc[2685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:40 dist-test-slave-9gcw krb5kdc[2685](info): set up 2 sockets
Aug 08 10:34:40 dist-test-slave-9gcw krb5kdc[2685](info): commencing operation
krb5kdc: starting...
W20260808 10:34:41.541448 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:41 dist-test-slave-9gcw krb5kdc[2685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:34:41.681733 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:46199
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:46199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:41.802261 2700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:41.802603 2700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:41.802693 2700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:41.806735 2700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:41.806842 2700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:41.806880 2700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:41.806918 2700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:41.806950 2700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:41.812623 2700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:46199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:46199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:41.813813 2700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:41.814917 2700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:41.824060 2706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:41.824065 2708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:41.824069 2705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:41.824463 2700 server_base.cc:1034] running on GCE node
I20260808 10:34:41.825205 2700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:41.826365 2700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:41.827669 2700 hybrid_clock.cc:648] HybridClock initialized: now 1786185281827566 us; error 54 us; skew 500 ppm
Aug 08 10:34:41 dist-test-slave-9gcw krb5kdc[2685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185281, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:41.830842 2700 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:34:41.831784 2700 webserver.cc:460] Webserver started at http://127.30.146.190:42213/ using document root <none> and password file <none>
I20260808 10:34:41.832383 2700 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:41.832448 2700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:41.832679 2700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:41.834867 2700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "55652a9ea63b4113a879e88a91d3e564"
format_stamp: "Formatted at 2026-08-08 10:34:41 on dist-test-slave-9gcw"
server_key: "aa7d380eda180af80919ad4674f516e0"
server_key_iv: "5be9d7eee67190a384d890033d1493c8"
server_key_version: "encryptionkey@0"
I20260808 10:34:41.835423 2700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "55652a9ea63b4113a879e88a91d3e564"
format_stamp: "Formatted at 2026-08-08 10:34:41 on dist-test-slave-9gcw"
server_key: "aa7d380eda180af80919ad4674f516e0"
server_key_iv: "5be9d7eee67190a384d890033d1493c8"
server_key_version: "encryptionkey@0"
I20260808 10:34:41.839401 2700 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:34:41.842319 2716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:41.843958 2700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:34:41.844076 2700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "55652a9ea63b4113a879e88a91d3e564"
format_stamp: "Formatted at 2026-08-08 10:34:41 on dist-test-slave-9gcw"
server_key: "aa7d380eda180af80919ad4674f516e0"
server_key_iv: "5be9d7eee67190a384d890033d1493c8"
server_key_version: "encryptionkey@0"
I20260808 10:34:41.844175 2700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:41.862212 2700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:41.865561 2700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:41.865734 2700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:41.873584 2700 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:46199
I20260808 10:34:41.873610 2773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:46199 every 8 connection(s)
I20260808 10:34:41.874614 2700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:41.877517 2774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:41.880013 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 2700
I20260808 10:34:41.880115 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:41.880456 31306 external_mini_cluster.cc:1428] Setting key 80571224f03220d22333876c5edf3cca
I20260808 10:34:41.883811 2774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564: Bootstrap starting.
I20260808 10:34:41.886281 2774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:41.887053 2774 log.cc:824] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:41.889559 2774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564: No bootstrap required, opened a new log
Aug 08 10:34:41 dist-test-slave-9gcw krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:41.892602 2774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55652a9ea63b4113a879e88a91d3e564" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46199 } }
I20260808 10:34:41.892895 2774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:41.892985 2774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55652a9ea63b4113a879e88a91d3e564, State: Initialized, Role: FOLLOWER
I20260808 10:34:41.893589 2774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55652a9ea63b4113a879e88a91d3e564" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46199 } }
I20260808 10:34:41.893783 2774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:41.893862 2774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:41.893965 2774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:41.895114 2774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55652a9ea63b4113a879e88a91d3e564" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46199 } }
I20260808 10:34:41.895536 2774 leader_election.cc:302] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55652a9ea63b4113a879e88a91d3e564; no voters:
I20260808 10:34:41.895905 2774 leader_election.cc:288] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:41.896026 2779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:41.896292 2779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [term 1 LEADER]: Becoming Leader. State: Replica: 55652a9ea63b4113a879e88a91d3e564, State: Running, Role: LEADER
I20260808 10:34:41.896698 2779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55652a9ea63b4113a879e88a91d3e564" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46199 } }
I20260808 10:34:41.897063 2777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:41.881984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.30.146.190:46199)
0808 10:34:41.882497 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:41.882513 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:41.882552 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:34:41.883227 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:41.883246 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:41.883585 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:41.883966 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:41.884832 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:41.885951 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:41.886908 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:41.887474 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:41.887665 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:41.890632 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:41.890639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:41.890657 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:41.890692 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:41.893243 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:41.893855 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:41.893861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:41.893868 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:41.894025 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:41.894433 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:41.894436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:41.894438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:41.894905 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:41.895084 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:41.895498 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 10:34:41.895870 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":150,"threads_started":1}
I20260808 10:34:41.897330 2774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:41.897990 2779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55652a9ea63b4113a879e88a91d3e564. Latest consensus state: current_term: 1 leader_uuid: "55652a9ea63b4113a879e88a91d3e564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55652a9ea63b4113a879e88a91d3e564" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46199 } } }
I20260808 10:34:41.898111 2779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:41.899363 2786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:41.899886 2780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55652a9ea63b4113a879e88a91d3e564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55652a9ea63b4113a879e88a91d3e564" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46199 } } }
I20260808 10:34:41.900306 2780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:41.902611 2786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:41.908680 2786 catalog_manager.cc:1347] Generated new cluster ID: 0533595c2fb64e4bb181ca141750feef
I20260808 10:34:41.908777 2786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:41.928936 2786 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:41.930307 2786 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:41.936108 2786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55652a9ea63b4113a879e88a91d3e564: Generated new TSK 0
I20260808 10:34:41.936980 2786 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:34:41.982156 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46199
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:42.101109 2802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:42.101437 2802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:42.101513 2802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:42.105599 2802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:42.105700 2802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:42.105792 2802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:42.111307 2802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:42.112418 2802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:42.113535 2802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:42.123510 2808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:42.123519 2810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:42.123680 2802 server_base.cc:1034] running on GCE node
W20260808 10:34:42.123525 2807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:42.124204 2802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:42.124858 2802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:42.126024 2802 hybrid_clock.cc:648] HybridClock initialized: now 1786185282125986 us; error 60 us; skew 500 ppm
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[2685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185282, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:42.129218 2802 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:34:42.130188 2802 webserver.cc:460] Webserver started at http://127.30.146.129:42645/ using document root <none> and password file <none>
I20260808 10:34:42.130795 2802 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:42.130870 2802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:42.131103 2802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:42.133358 2802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "4a5b7d68a84046ebbf05462de62a1bf9"
format_stamp: "Formatted at 2026-08-08 10:34:42 on dist-test-slave-9gcw"
server_key: "3c6832150ccc6dc998a01ddce17b5d31"
server_key_iv: "2aabf9e34e76581cccd8645b96ee6785"
server_key_version: "encryptionkey@0"
I20260808 10:34:42.133989 2802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "4a5b7d68a84046ebbf05462de62a1bf9"
format_stamp: "Formatted at 2026-08-08 10:34:42 on dist-test-slave-9gcw"
server_key: "3c6832150ccc6dc998a01ddce17b5d31"
server_key_iv: "2aabf9e34e76581cccd8645b96ee6785"
server_key_version: "encryptionkey@0"
I20260808 10:34:42.138104 2802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:42.140771 2818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:42.142367 2802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:34:42.142524 2802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "4a5b7d68a84046ebbf05462de62a1bf9"
format_stamp: "Formatted at 2026-08-08 10:34:42 on dist-test-slave-9gcw"
server_key: "3c6832150ccc6dc998a01ddce17b5d31"
server_key_iv: "2aabf9e34e76581cccd8645b96ee6785"
server_key_version: "encryptionkey@0"
I20260808 10:34:42.142652 2802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:42.168704 2802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:42.172281 2802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:42.172518 2802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:42.173184 2802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:42.174069 2802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:42.174147 2802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:42.174221 2802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:42.174266 2802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:42.184068 2802 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:40419
I20260808 10:34:42.184099 2938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:40419 every 8 connection(s)
I20260808 10:34:42.185025 2802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:42.190553 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 2802
I20260808 10:34:42.190708 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:42.191018 31306 external_mini_cluster.cc:1428] Setting key 1642183f26e647e3b28a37f6cb51771b
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185282, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:42.201644 2777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:42.186985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:54665 (local address 127.30.146.190:46199)
0808 10:34:42.187133 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:42.187138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:42.188012 (+ 874us) server_negotiation.cc:407] Connection header received
0808 10:34:42.189573 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:42.189576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:42.189650 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:42.189766 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:42.191080 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.191805 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.193678 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.193957 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.194077 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:42.197042 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:42.197046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:42.197048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:42.197073 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:42.198977 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:42.199722 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:42.199726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:42.199727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:42.199878 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:42.200372 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:42.200377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:42.200379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:42.200604 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:42.200721 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:42.201326 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 10:34:42.201469 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:34:42.202546 2941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:42.187321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46199 (local address 127.30.146.129:54665)
0808 10:34:42.187834 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:42.187880 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:42.189352 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:42.189949 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:42.189956 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:42.190335 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:42.190921 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:42.190937 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.191991 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.191994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:42.193463 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:42.193479 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.194147 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.194154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:42.194404 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:42.195210 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:42.195228 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:42.196845 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:42.199110 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:42.199122 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:42.199139 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:42.199576 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:42.200028 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:42.200031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:42.200033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:42.200254 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:42.200722 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:42.200730 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:42.200996 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 10:34:42.201279 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 10:34:42.201656 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":126,"threads_started":1}
I20260808 10:34:42.203912 2939 heartbeater.cc:347] Connected to a master server at 127.30.146.190:46199
I20260808 10:34:42.204186 2939 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:42.204742 2939 heartbeater.cc:511] Master 127.30.146.190:46199 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:34:42.206792 2735 ts_manager.cc:194] Registered new tserver with Master: 4a5b7d68a84046ebbf05462de62a1bf9 (127.30.146.129:40419)
I20260808 10:34:42.208767 2735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:54665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:34:42.225468 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46199
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:42.348280 2944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:42.348608 2944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:42.348698 2944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:42.352864 2944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:42.352936 2944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:42.352995 2944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:42.358512 2944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:42.359557 2944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:42.360540 2944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:42.370841 2950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:42.370859 2949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:42.370841 2952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:42.371042 2944 server_base.cc:1034] running on GCE node
I20260808 10:34:42.371531 2944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:42.372190 2944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:42.373387 2944 hybrid_clock.cc:648] HybridClock initialized: now 1786185282373358 us; error 47 us; skew 500 ppm
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[2685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185282, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:42.376315 2944 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:34:42.377357 2944 webserver.cc:460] Webserver started at http://127.30.146.130:45543/ using document root <none> and password file <none>
I20260808 10:34:42.377946 2944 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:42.378046 2944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:42.378276 2944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:42.380358 2944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "2b9a588c906349f8a483bf9810024fbf"
format_stamp: "Formatted at 2026-08-08 10:34:42 on dist-test-slave-9gcw"
server_key: "9e8e23e74cc800efa1d60dcb2f20118e"
server_key_iv: "09662f1c7b86bc4505087aa0bdbc8b17"
server_key_version: "encryptionkey@0"
I20260808 10:34:42.380942 2944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "2b9a588c906349f8a483bf9810024fbf"
format_stamp: "Formatted at 2026-08-08 10:34:42 on dist-test-slave-9gcw"
server_key: "9e8e23e74cc800efa1d60dcb2f20118e"
server_key_iv: "09662f1c7b86bc4505087aa0bdbc8b17"
server_key_version: "encryptionkey@0"
I20260808 10:34:42.384773 2944 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:34:42.387395 2959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:42.388836 2944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:42.388978 2944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "2b9a588c906349f8a483bf9810024fbf"
format_stamp: "Formatted at 2026-08-08 10:34:42 on dist-test-slave-9gcw"
server_key: "9e8e23e74cc800efa1d60dcb2f20118e"
server_key_iv: "09662f1c7b86bc4505087aa0bdbc8b17"
server_key_version: "encryptionkey@0"
I20260808 10:34:42.389096 2944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:42.411031 2944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:42.414196 2944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:42.414408 2944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:42.415036 2944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:42.415899 2944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:42.415946 2944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:42.416016 2944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:42.416049 2944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:42.424998 2944 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:39661
I20260808 10:34:42.425024 3076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:39661 every 8 connection(s)
I20260808 10:34:42.425956 2944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:42.433490 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 2944
I20260808 10:34:42.433626 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:42.433974 31306 external_mini_cluster.cc:1428] Setting key b4a409cd66e22ac58bfc27e1050a3ba4
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185282, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:42.442052 2777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:42.427904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:43903 (local address 127.30.146.190:46199)
0808 10:34:42.428054 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:42.428058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:42.428894 (+ 836us) server_negotiation.cc:407] Connection header received
0808 10:34:42.430488 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:42.430492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:42.430555 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:42.430651 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:42.432127 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.432690 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.434302 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.434639 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.434729 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:42.438093 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:42.438096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:42.438098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:42.438125 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:42.439652 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:42.440231 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:42.440239 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:42.440242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:42.440331 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:42.440736 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:42.440739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:42.440741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:42.440946 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:42.441071 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:42.441811 (+ 740us) server_negotiation.cc:299] Negotiation successful
0808 10:34:42.441925 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:34:42.443398 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:42.428279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46199 (local address 127.30.146.130:43903)
0808 10:34:42.428725 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:42.428770 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:42.430218 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:42.430914 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:42.430926 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:42.431469 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:42.431977 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:42.431991 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.432826 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.432829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:42.434149 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:42.434161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.434808 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.434814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:42.434977 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:42.435791 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:42.435818 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:42.437882 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:42.439799 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:42.439805 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:42.439831 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:42.440110 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:42.440460 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:42.440464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:42.440466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:42.440622 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:42.441060 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:42.441067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:42.441473 (+ 406us) client_negotiation.cc:769] Sending connection context
0808 10:34:42.441726 (+ 253us) client_negotiation.cc:240] Negotiation successful
0808 10:34:42.442166 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":104,"threads_started":1}
I20260808 10:34:42.444736 3077 heartbeater.cc:347] Connected to a master server at 127.30.146.190:46199
I20260808 10:34:42.445024 3077 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:34:42.445678 3077 heartbeater.cc:511] Master 127.30.146.190:46199 requested a full tablet report, sending...
I20260808 10:34:42.446828 2735 ts_manager.cc:194] Registered new tserver with Master: 2b9a588c906349f8a483bf9810024fbf (127.30.146.130:39661)
I20260808 10:34:42.447598 2735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:43903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:34:42.466641 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46199
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:42.595592 3082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:42.595922 3082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:42.595981 3082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:42.601191 3082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:42.601284 3082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:42.601356 3082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:42.607599 3082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:42.609032 3082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:42.610262 3082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:42.621088 3088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:42.621088 3091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:42.621088 3089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:42.621516 3082 server_base.cc:1034] running on GCE node
I20260808 10:34:42.622013 3082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:42.622740 3082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:42.623922 3082 hybrid_clock.cc:648] HybridClock initialized: now 1786185282623907 us; error 25 us; skew 500 ppm
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[2685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185282, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:42.627015 3082 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:34:42.628340 3082 webserver.cc:460] Webserver started at http://127.30.146.131:40155/ using document root <none> and password file <none>
I20260808 10:34:42.629285 3082 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:42.629396 3082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:42.629666 3082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:42.632207 3082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "40bd6658d0974d1eb974e0ead7bfea60"
format_stamp: "Formatted at 2026-08-08 10:34:42 on dist-test-slave-9gcw"
server_key: "977ae7ae4372df54f47c270bfadba521"
server_key_iv: "6740945df3f1e9e7c6626d652a024875"
server_key_version: "encryptionkey@0"
I20260808 10:34:42.632993 3082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "40bd6658d0974d1eb974e0ead7bfea60"
format_stamp: "Formatted at 2026-08-08 10:34:42 on dist-test-slave-9gcw"
server_key: "977ae7ae4372df54f47c270bfadba521"
server_key_iv: "6740945df3f1e9e7c6626d652a024875"
server_key_version: "encryptionkey@0"
I20260808 10:34:42.637641 3082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:34:42.640590 3100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:42.641973 3082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:42.642122 3082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "40bd6658d0974d1eb974e0ead7bfea60"
format_stamp: "Formatted at 2026-08-08 10:34:42 on dist-test-slave-9gcw"
server_key: "977ae7ae4372df54f47c270bfadba521"
server_key_iv: "6740945df3f1e9e7c6626d652a024875"
server_key_version: "encryptionkey@0"
I20260808 10:34:42.642249 3082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:42.663739 3082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:42.667397 3082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:42.667622 3082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:42.668288 3082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:42.669198 3082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:42.669271 3082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:42.669341 3082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:42.669373 3082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:42.679186 3082 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:34813
I20260808 10:34:42.679214 3223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:34813 every 8 connection(s)
I20260808 10:34:42.680083 3082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:42.685608 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 3082
I20260808 10:34:42.685784 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:42.686084 31306 external_mini_cluster.cc:1428] Setting key bd50cd846958f57ede560d21d0f18f0b
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185282, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:42.696341 2777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:42.681992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:51217 (local address 127.30.146.190:46199)
0808 10:34:42.682155 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:42.682159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:42.683004 (+ 845us) server_negotiation.cc:407] Connection header received
0808 10:34:42.684837 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:42.684840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:42.684906 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:42.685016 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:42.686435 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.687424 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.688883 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.689119 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.689259 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:42.692300 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:42.692304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:42.692307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:42.692342 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:42.693918 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:42.694462 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:42.694467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:42.694469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:42.694553 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:42.694923 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:42.694926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:42.694928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:42.695336 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:42.695472 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:42.696032 (+ 560us) server_negotiation.cc:299] Negotiation successful
0808 10:34:42.696166 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:34:42.697346 3226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:42.682301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46199 (local address 127.30.146.131:51217)
0808 10:34:42.682822 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:42.682871 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:42.684592 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:42.685227 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:42.685235 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:42.685597 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:42.686254 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:42.686274 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.687559 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.687563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:42.688751 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:42.688761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.689314 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.689319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:42.689505 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:42.690088 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:42.690106 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:42.692114 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:42.694048 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:42.694054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:42.694070 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:42.694350 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:42.694656 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:42.694661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:42.694665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:42.694816 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:42.695482 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:42.695491 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:42.695774 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 10:34:42.695987 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 10:34:42.696384 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":116,"threads_started":1}
I20260808 10:34:42.698782 3224 heartbeater.cc:347] Connected to a master server at 127.30.146.190:46199
I20260808 10:34:42.699085 3224 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:42.699707 3224 heartbeater.cc:511] Master 127.30.146.190:46199 requested a full tablet report, sending...
I20260808 10:34:42.700925 2735 ts_manager.cc:194] Registered new tserver with Master: 40bd6658d0974d1eb974e0ead7bfea60 (127.30.146.131:34813)
I20260808 10:34:42.701548 2735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:51217
I20260808 10:34:42.712816 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[2685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:34:42.742736 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:42.731898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.30.146.129:40419)
0808 10:34:42.732272 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:42.732278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:42.732303 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:34:42.732360 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:42.732365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:42.732550 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:42.732773 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:42.733547 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.734409 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.735380 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.735879 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.735977 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:42.739336 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:42.739346 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:42.739359 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:42.739385 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:42.740954 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:42.741503 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:42.741508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:42.741509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:42.741593 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:42.741859 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:42.741861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:42.741862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:42.742104 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:42.742292 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:42.742464 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 10:34:42.742589 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":146,"threads_started":1}
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:42.755874 2777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:42.746007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55444 (local address 127.30.146.190:46199)
0808 10:34:42.746187 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:42.746195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:42.746219 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:34:42.746492 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:42.746496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:42.746558 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:42.746658 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:42.747416 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.748059 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.749065 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:42.749420 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:42.749546 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:42.752312 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:42.752316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:42.752319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:42.752357 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:42.753930 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:42.754451 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:42.754455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:42.754457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:42.754547 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:42.754842 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:42.754845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:42.754847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:42.755119 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:42.755277 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:42.755550 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 10:34:42.755691 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 10:34:42.756340 2735 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55444
I20260808 10:34:42.757272 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 2802
I20260808 10:34:42.767284 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 2944
I20260808 10:34:42.774600 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 3082
I20260808 10:34:42.781896 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 2700
2026-08-08T10:34:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2347 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[3237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[3237](info): set up 2 sockets
Aug 08 10:34:42 dist-test-slave-9gcw krb5kdc[3237](info): commencing operation
krb5kdc: starting...
W20260808 10:34:43.888989 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:43 dist-test-slave-9gcw krb5kdc[3237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:34:44.028347 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:35393
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:35393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:44.144942 3250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:44.145319 3250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:44.145414 3250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:44.149509 3250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:44.149621 3250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:44.149660 3250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:44.149698 3250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:44.149739 3250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:44.155092 3250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:35393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:35393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:44.156299 3250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:44.157445 3250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:44.166611 3260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:44.166666 3257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:44.166611 3258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:44.166992 3250 server_base.cc:1034] running on GCE node
I20260808 10:34:44.167681 3250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:44.168872 3250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:44.170074 3250 hybrid_clock.cc:648] HybridClock initialized: now 1786185284170042 us; error 44 us; skew 500 ppm
Aug 08 10:34:44 dist-test-slave-9gcw krb5kdc[3237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185284, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:44.178267 3250 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:34:44.179118 3250 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:44.179239 3250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:44.179502 3250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:44.181919 3250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "5e7769eb0ba8457eb588a57efd526924"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "42c80959f2fb217c56be0fa0014ae82e"
server_key_iv: "12650bef7a193d241a10b7d6d1689103"
server_key_version: "encryptionkey@0"
I20260808 10:34:44.182569 3250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "5e7769eb0ba8457eb588a57efd526924"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "42c80959f2fb217c56be0fa0014ae82e"
server_key_iv: "12650bef7a193d241a10b7d6d1689103"
server_key_version: "encryptionkey@0"
I20260808 10:34:44.186691 3250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:44.189448 3267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:44.190758 3250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 10:34:44.190919 3250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "5e7769eb0ba8457eb588a57efd526924"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "42c80959f2fb217c56be0fa0014ae82e"
server_key_iv: "12650bef7a193d241a10b7d6d1689103"
server_key_version: "encryptionkey@0"
I20260808 10:34:44.191048 3250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:44.211555 3250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:44.214926 3250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:44.215162 3250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:44.223033 3250 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:35393
I20260808 10:34:44.223327 3327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:35393 every 8 connection(s)
I20260808 10:34:44.223791 3250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:44.226608 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 3250
I20260808 10:34:44.226758 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:44.226830 3328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:44.227135 31306 external_mini_cluster.cc:1428] Setting key 68e22373d8d10b567c94258a2b60c204
I20260808 10:34:44.234208 3328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924: Bootstrap starting.
Aug 08 10:34:44 dist-test-slave-9gcw krb5kdc[3237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:44.236792 3328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:44.237659 3328 log.cc:824] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:44.240160 3328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924: No bootstrap required, opened a new log
I20260808 10:34:44.243235 3331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:44.228594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35154 (local address 127.30.146.190:35393)
0808 10:34:44.229041 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:44.229058 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:44.229095 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:34:44.229829 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:44.229847 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:44.230217 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:44.230520 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:44.231487 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:44.232621 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:44.233838 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:44.234408 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:44.234592 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:44.237500 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:44.237505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:44.237519 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:44.237548 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:44.239752 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:44.240256 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:44.240262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:44.240268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:44.240424 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:44.240764 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:44.240767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:44.240769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:44.241339 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:44.241522 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:44.241821 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 10:34:44.242139 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":144,"threads_started":1}
I20260808 10:34:44.244956 3328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7769eb0ba8457eb588a57efd526924" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35393 } }
I20260808 10:34:44.245253 3328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:44.245321 3328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e7769eb0ba8457eb588a57efd526924, State: Initialized, Role: FOLLOWER
I20260808 10:34:44.245782 3328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7769eb0ba8457eb588a57efd526924" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35393 } }
I20260808 10:34:44.245939 3328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:44.246033 3328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:44.246140 3328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:44.247128 3328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7769eb0ba8457eb588a57efd526924" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35393 } }
I20260808 10:34:44.247481 3328 leader_election.cc:302] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e7769eb0ba8457eb588a57efd526924; no voters:
I20260808 10:34:44.247826 3328 leader_election.cc:288] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:44.248073 3333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:44.248426 3333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [term 1 LEADER]: Becoming Leader. State: Replica: 5e7769eb0ba8457eb588a57efd526924, State: Running, Role: LEADER
I20260808 10:34:44.248903 3333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7769eb0ba8457eb588a57efd526924" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35393 } }
I20260808 10:34:44.249176 3328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:44.250617 3335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e7769eb0ba8457eb588a57efd526924. Latest consensus state: current_term: 1 leader_uuid: "5e7769eb0ba8457eb588a57efd526924" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7769eb0ba8457eb588a57efd526924" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35393 } } }
I20260808 10:34:44.250759 3335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:44.251084 3334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e7769eb0ba8457eb588a57efd526924" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e7769eb0ba8457eb588a57efd526924" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35393 } } }
I20260808 10:34:44.251238 3334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:44.251258 3343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:44.254287 3343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:44.260241 3343 catalog_manager.cc:1347] Generated new cluster ID: 3e348c87b8c644a6a1e74ce8e0c65e8d
I20260808 10:34:44.260334 3343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:44.276988 3343 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:44.278095 3343 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:44.284694 3343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924: Generated new TSK 0
I20260808 10:34:44.285529 3343 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:34:44.324777 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:35393
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:44.457923 3356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:44.458290 3356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:44.458375 3356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:44.462612 3356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:44.462733 3356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:44.462832 3356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:44.468482 3356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:35393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:44.469767 3356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:44.470955 3356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:44.481845 3361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:44.482045 3356 server_base.cc:1034] running on GCE node
W20260808 10:34:44.482295 3364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:44.482275 3362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:44.482849 3356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:44.483704 3356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:44.484943 3356 hybrid_clock.cc:648] HybridClock initialized: now 1786185284484923 us; error 42 us; skew 500 ppm
Aug 08 10:34:44 dist-test-slave-9gcw krb5kdc[3237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185284, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:44.488317 3356 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:34:44.489245 3356 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:44.489357 3356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:44.489645 3356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:44.492054 3356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "4c776a4e622b4151bdd5126b933e9ca0"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "82c9e3ca04f80bfc665593f53f0b8605"
server_key_iv: "8543bca006b8f90ea5bafc3a6d0ed05e"
server_key_version: "encryptionkey@0"
I20260808 10:34:44.492730 3356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "4c776a4e622b4151bdd5126b933e9ca0"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "82c9e3ca04f80bfc665593f53f0b8605"
server_key_iv: "8543bca006b8f90ea5bafc3a6d0ed05e"
server_key_version: "encryptionkey@0"
I20260808 10:34:44.497417 3356 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 10:34:44.500203 3370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:44.501982 3356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:44.502157 3356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "4c776a4e622b4151bdd5126b933e9ca0"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "82c9e3ca04f80bfc665593f53f0b8605"
server_key_iv: "8543bca006b8f90ea5bafc3a6d0ed05e"
server_key_version: "encryptionkey@0"
I20260808 10:34:44.502305 3356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:44.519026 3356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:44.522730 3356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:44.522965 3356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:44.523639 3356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:44.524626 3356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:44.524681 3356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:44.524736 3356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:44.524755 3356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:44.535689 3356 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:39891
I20260808 10:34:44.535708 3489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:39891 every 8 connection(s)
I20260808 10:34:44.536396 3356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:44.544345 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 3356
I20260808 10:34:44.544481 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:44.544842 31306 external_mini_cluster.cc:1428] Setting key a8e3c9e02ed221d64c7fb9df1521ac2f
Aug 08 10:34:44 dist-test-slave-9gcw krb5kdc[3237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185284, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:44.553331 3331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:44.538468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:43697 (local address 127.30.146.190:35393)
0808 10:34:44.538639 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:44.538644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:44.539748 (+ 1104us) server_negotiation.cc:407] Connection header received
0808 10:34:44.541279 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:44.541282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:44.541338 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:44.541427 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:44.542751 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:44.543363 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:44.544689 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:44.544935 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:44.545067 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:44.548456 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:44.548463 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:44.548469 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:44.548506 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:44.550545 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:44.551328 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:44.551334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:44.551339 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:44.551439 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:44.551907 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:44.551913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:44.551918 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:44.552191 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:44.552300 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:44.552936 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 10:34:44.553096 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:34:44.554325 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:44.539030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35393 (local address 127.30.146.129:43697)
0808 10:34:44.539565 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:44.539616 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:44.540961 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:44.541640 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:44.541648 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:44.542052 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:44.542580 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:44.542593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:44.543491 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:44.543494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:44.544526 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:44.544537 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:44.545140 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:44.545174 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:44.545413 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:44.546156 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:44.546183 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:44.548274 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:44.550689 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:44.550698 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:44.550719 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:44.551213 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:44.551543 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:44.551546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:44.551547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:44.551795 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:44.552317 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:44.552323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:44.552598 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 10:34:44.552852 (+ 254us) client_negotiation.cc:240] Negotiation successful
0808 10:34:44.553226 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":125,"threads_started":1}
I20260808 10:34:44.555873 3490 heartbeater.cc:347] Connected to a master server at 127.30.146.190:35393
I20260808 10:34:44.556207 3490 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:44.556838 3490 heartbeater.cc:511] Master 127.30.146.190:35393 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:34:44.558750 3286 ts_manager.cc:194] Registered new tserver with Master: 4c776a4e622b4151bdd5126b933e9ca0 (127.30.146.129:39891)
I20260808 10:34:44.559968 3286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:43697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:34:44.580475 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:35393
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:44.704814 3495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:44.705389 3495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:44.705542 3495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:44.713044 3495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:44.713181 3495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:44.713310 3495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:44.721756 3495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:35393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:44.723547 3495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:44.724704 3495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:44.738024 3501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:44.738027 3503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:44.738076 3500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:44.738601 3495 server_base.cc:1034] running on GCE node
I20260808 10:34:44.739068 3495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:44.739763 3495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:44.740963 3495 hybrid_clock.cc:648] HybridClock initialized: now 1786185284740931 us; error 50 us; skew 500 ppm
Aug 08 10:34:44 dist-test-slave-9gcw krb5kdc[3237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185284, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:44.744160 3495 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:34:44.744927 3495 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:44.745026 3495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:44.745324 3495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:44.747496 3495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "999e1cfd1f21408ea90cb491630dba66"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "31983b44d653958b532098c23602139e"
server_key_iv: "d3011a3bd5ef74492eaa6948e4db1ccd"
server_key_version: "encryptionkey@0"
I20260808 10:34:44.748097 3495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "999e1cfd1f21408ea90cb491630dba66"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "31983b44d653958b532098c23602139e"
server_key_iv: "d3011a3bd5ef74492eaa6948e4db1ccd"
server_key_version: "encryptionkey@0"
I20260808 10:34:44.752279 3495 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:34:44.754768 3509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:44.756054 3495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:34:44.756206 3495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "999e1cfd1f21408ea90cb491630dba66"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "31983b44d653958b532098c23602139e"
server_key_iv: "d3011a3bd5ef74492eaa6948e4db1ccd"
server_key_version: "encryptionkey@0"
I20260808 10:34:44.756323 3495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:44.776082 3495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:44.779160 3495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:44.779377 3495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:44.780059 3495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:44.780947 3495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:44.781020 3495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:44.781093 3495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:44.781148 3495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:44.790668 3495 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:42603
I20260808 10:34:44.790688 3632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:42603 every 8 connection(s)
I20260808 10:34:44.791320 3495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:44.799427 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 3495
I20260808 10:34:44.799585 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:44.799901 31306 external_mini_cluster.cc:1428] Setting key 1bb2116efc79bfa1790ab2e81c2839b4
Aug 08 10:34:44 dist-test-slave-9gcw krb5kdc[3237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185284, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:44.805938 3331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:44.793207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:37055 (local address 127.30.146.190:35393)
0808 10:34:44.793367 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:44.793371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:44.794253 (+ 882us) server_negotiation.cc:407] Connection header received
0808 10:34:44.795775 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:44.795778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:44.795838 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:44.795919 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:44.797222 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:44.797771 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:44.798935 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:44.799184 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:44.799299 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:44.802186 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:44.802190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:44.802193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:44.802227 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:44.803718 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:44.804260 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:44.804264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:44.804266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:44.804353 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:44.804729 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:44.804732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:44.804734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:44.804930 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:44.805040 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:44.805630 (+ 590us) server_negotiation.cc:299] Negotiation successful
0808 10:34:44.805764 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:34:44.806798 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:44.793608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35393 (local address 127.30.146.130:37055)
0808 10:34:44.794073 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:44.794116 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:44.795560 (+ 1444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:44.796111 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:44.796122 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:44.796546 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:44.797050 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:44.797063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:44.797920 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:44.797928 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:44.798802 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:44.798812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:44.799353 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:44.799357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:44.799532 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:44.800272 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:44.800289 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:44.802011 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:44.803856 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:44.803862 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:44.803878 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:44.804144 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:44.804448 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:44.804452 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:44.804454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:44.804601 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:44.805040 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:44.805048 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:44.805380 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 10:34:44.805568 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 10:34:44.805910 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":115,"threads_started":1}
I20260808 10:34:44.808115 3633 heartbeater.cc:347] Connected to a master server at 127.30.146.190:35393
I20260808 10:34:44.808403 3633 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:44.808990 3633 heartbeater.cc:511] Master 127.30.146.190:35393 requested a full tablet report, sending...
I20260808 10:34:44.810247 3286 ts_manager.cc:194] Registered new tserver with Master: 999e1cfd1f21408ea90cb491630dba66 (127.30.146.130:42603)
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:34:44.811087 3286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:37055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:34:44.833050 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:35393
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:44.961035 3638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:44.961416 3638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:44.961503 3638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:44.966297 3638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:44.966370 3638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:44.966439 3638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:44.972841 3638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:35393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:44.974514 3638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:44.975708 3638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:44.986747 3644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:44.986701 3645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:44.986840 3638 server_base.cc:1034] running on GCE node
W20260808 10:34:44.986752 3647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:44.987552 3638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:44.988442 3638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:44.989840 3638 hybrid_clock.cc:648] HybridClock initialized: now 1786185284989812 us; error 49 us; skew 500 ppm
Aug 08 10:34:44 dist-test-slave-9gcw krb5kdc[3237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185284, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:44.992868 3638 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:34:44.993728 3638 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:44.993829 3638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:44.994076 3638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:44.996338 3638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "37a609b9615240c48df9ccc74a2b7695"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "2d9a35b209c0bfc1d3e6c1dac0c4caf0"
server_key_iv: "634f05b2ee507c56fbe1acacbd2fb74c"
server_key_version: "encryptionkey@0"
I20260808 10:34:44.996950 3638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "37a609b9615240c48df9ccc74a2b7695"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "2d9a35b209c0bfc1d3e6c1dac0c4caf0"
server_key_iv: "634f05b2ee507c56fbe1acacbd2fb74c"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.001009 3638 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:34:45.003985 3655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:45.005432 3638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:45.005571 3638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "37a609b9615240c48df9ccc74a2b7695"
format_stamp: "Formatted at 2026-08-08 10:34:44 on dist-test-slave-9gcw"
server_key: "2d9a35b209c0bfc1d3e6c1dac0c4caf0"
server_key_iv: "634f05b2ee507c56fbe1acacbd2fb74c"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.005697 3638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:45.021588 3638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:45.024673 3638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:45.024900 3638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:45.025537 3638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:45.026412 3638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:45.026487 3638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:45.026561 3638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:45.026614 3638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:45.036190 3638 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:33007
I20260808 10:34:45.036229 3786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:33007 every 8 connection(s)
I20260808 10:34:45.036854 3638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:45.041882 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 3638
I20260808 10:34:45.042059 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:45.042424 31306 external_mini_cluster.cc:1428] Setting key 07b01f9823ea95ebf9ccebf0eaeee0da
Aug 08 10:34:45 dist-test-slave-9gcw krb5kdc[3237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185284, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:45.052726 3331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.038949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:41529 (local address 127.30.146.190:35393)
0808 10:34:45.039112 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.039116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.040116 (+ 1000us) server_negotiation.cc:407] Connection header received
0808 10:34:45.041766 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.041769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.041828 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.041916 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:45.043735 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.044542 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.045935 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.046150 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.046245 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.048809 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:45.048811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:45.048813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:45.048840 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:45.050592 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:45.051115 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:45.051118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:45.051120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:45.051213 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:45.051621 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:45.051624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:45.051625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:45.051831 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:45.051923 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.052443 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.052558 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:34:45.053625 3789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.039370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35393 (local address 127.30.146.131:41529)
0808 10:34:45.039923 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:45.039970 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:45.041453 (+ 1483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:45.042199 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:45.042211 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:45.042820 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:45.043560 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.043582 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.044708 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.044711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.045789 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.045800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.046300 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.046303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.046450 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.047058 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:45.047075 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:45.048641 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:45.050697 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:45.050702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:45.050714 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:45.050987 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:45.051305 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:45.051309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:45.051310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:45.051523 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:45.051943 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:45.051948 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:45.052190 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 10:34:45.052358 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 10:34:45.052623 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":117,"threads_started":1}
I20260808 10:34:45.054816 3787 heartbeater.cc:347] Connected to a master server at 127.30.146.190:35393
I20260808 10:34:45.055088 3787 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:45.055716 3787 heartbeater.cc:511] Master 127.30.146.190:35393 requested a full tablet report, sending...
I20260808 10:34:45.056756 3285 ts_manager.cc:194] Registered new tserver with Master: 37a609b9615240c48df9ccc74a2b7695 (127.30.146.131:33007)
I20260808 10:34:45.057417 3285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:41529
I20260808 10:34:45.070240 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:34:45.080055 3331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.072288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35170 (local address 127.30.146.190:35393)
0808 10:34:45.072486 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.072491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.072610 (+ 119us) server_negotiation.cc:407] Connection header received
0808 10:34:45.072769 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.072773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.072828 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.072921 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:45.073722 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.074256 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.075154 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.075406 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.075521 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.077048 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:45.077051 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:45.077054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:45.077077 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:45.078564 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:45.078956 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:45.078959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:45.078960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:45.079038 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:45.079276 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:45.079278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:45.079279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:45.079509 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:45.079618 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.079830 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.079929 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 10:34:45.083386 3285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:34:45.086582 3285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:45.103128 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.096200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:42603 (local address 127.0.0.1:38594)
0808 10:34:45.097116 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:45.097133 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:45.097272 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:45.097969 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:45.097977 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:45.098016 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:45.098199 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.098207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.099616 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.099623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.101417 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.101428 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.102644 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.102646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.102728 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.102761 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:34:45.102848 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:34:45.102942 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":931,"spinlock_wait_cycles":6656,"thread_start_us":102,"threads_started":1}
I20260808 10:34:45.103231 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.096456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.30.146.130:42603)
0808 10:34:45.096889 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.096907 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.097390 (+ 483us) server_negotiation.cc:407] Connection header received
0808 10:34:45.097491 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.097499 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.097761 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.098370 (+ 609us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:45.098590 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.099470 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.101591 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.102500 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.102643 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.102792 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.102906 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.103041 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":133,"threads_started":1}
I20260808 10:34:45.104135 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.097062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:33007 (local address 127.0.0.1:49270)
0808 10:34:45.097893 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:45.097910 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:45.098023 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:45.098515 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:45.098518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:45.098536 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:45.098685 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.098692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.100022 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.100025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.101423 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.101430 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.103672 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.103675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.103788 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.103821 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:34:45.103913 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:34:45.103994 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":73,"spinlock_wait_cycles":6656,"thread_start_us":118,"threads_started":1}
I20260808 10:34:45.104579 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.096200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:39891 (local address 127.0.0.1:45714)
0808 10:34:45.096809 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:45.096867 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:45.097017 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:45.098157 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:45.098160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:45.098176 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:45.098272 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.098282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.099797 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.099800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.101402 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.101413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.104153 (+ 2740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.104156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.104298 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.104327 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 10:34:45.104400 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:34:45.104474 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":72,"thread_start_us":144,"threads_started":1}
I20260808 10:34:45.105028 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.097439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.30.146.131:33007)
0808 10:34:45.097917 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.097928 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.097983 (+ 55us) server_negotiation.cc:407] Connection header received
0808 10:34:45.098160 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.098166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.098363 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.098520 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:45.098815 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.099880 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.101543 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.102500 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.102650 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.102792 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.104750 (+ 1958us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.104873 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":174,"threads_started":1}
I20260808 10:34:45.105038 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.096360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45714 (local address 127.30.146.129:39891)
0808 10:34:45.097549 (+ 1189us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.097558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.097587 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:34:45.097673 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.097678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.097900 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.098085 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:45.098409 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.099658 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.102872 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.103821 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.103960 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.104091 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.104810 (+ 719us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.104914 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1076,"thread_start_us":102,"threads_started":1}
I20260808 10:34:45.106745 3561 tablet_service.cc:1467] Processing CreateTablet for tablet d5d7d10354d947db881ecf98a50357f2 (DEFAULT_TABLE table=test-table [id=fb570a60b0a34fc88c98e959dad19bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:45.107924 3561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5d7d10354d947db881ecf98a50357f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:45.110155 3711 tablet_service.cc:1467] Processing CreateTablet for tablet d5d7d10354d947db881ecf98a50357f2 (DEFAULT_TABLE table=test-table [id=fb570a60b0a34fc88c98e959dad19bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:45.110167 3421 tablet_service.cc:1467] Processing CreateTablet for tablet d5d7d10354d947db881ecf98a50357f2 (DEFAULT_TABLE table=test-table [id=fb570a60b0a34fc88c98e959dad19bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:45.111200 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5d7d10354d947db881ecf98a50357f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:45.111207 3421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5d7d10354d947db881ecf98a50357f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:45.116906 3806 tablet_bootstrap.cc:492] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66: Bootstrap starting.
I20260808 10:34:45.118346 3808 tablet_bootstrap.cc:492] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0: Bootstrap starting.
I20260808 10:34:45.118502 3807 tablet_bootstrap.cc:492] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: Bootstrap starting.
I20260808 10:34:45.120332 3808 tablet_bootstrap.cc:654] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:45.120404 3806 tablet_bootstrap.cc:654] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:45.120735 3807 tablet_bootstrap.cc:654] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:45.121259 3808 log.cc:824] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:45.122068 3806 log.cc:824] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:45.122246 3807 log.cc:824] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:45.123461 3808 tablet_bootstrap.cc:492] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0: No bootstrap required, opened a new log
I20260808 10:34:45.123678 3808 ts_tablet_manager.cc:1397] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:34:45.124392 3806 tablet_bootstrap.cc:492] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66: No bootstrap required, opened a new log
I20260808 10:34:45.124614 3806 ts_tablet_manager.cc:1397] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 10:34:45.125021 3807 tablet_bootstrap.cc:492] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: No bootstrap required, opened a new log
I20260808 10:34:45.125257 3807 ts_tablet_manager.cc:1397] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260808 10:34:45.127744 3808 raft_consensus.cc:348] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 } } peers { permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 } } peers { permanent_uuid: "37a609b9615240c48df9ccc74a2b7695" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 33007 } }
I20260808 10:34:45.128012 3808 raft_consensus.cc:374] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:45.128087 3808 raft_consensus.cc:729] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c776a4e622b4151bdd5126b933e9ca0, State: Initialized, Role: FOLLOWER
I20260808 10:34:45.128561 3808 consensus_queue.cc:260] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 } } peers { permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 } } peers { permanent_uuid: "37a609b9615240c48df9ccc74a2b7695" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 33007 } }
I20260808 10:34:45.128979 3806 raft_consensus.cc:348] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 } } peers { permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 } } peers { permanent_uuid: "37a609b9615240c48df9ccc74a2b7695" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 33007 } }
I20260808 10:34:45.129297 3490 heartbeater.cc:503] Master 127.30.146.190:35393 was elected leader, sending a full tablet report...
I20260808 10:34:45.129372 3806 raft_consensus.cc:374] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:45.129436 3806 raft_consensus.cc:729] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 999e1cfd1f21408ea90cb491630dba66, State: Initialized, Role: FOLLOWER
I20260808 10:34:45.129436 3807 raft_consensus.cc:348] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 } } peers { permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 } } peers { permanent_uuid: "37a609b9615240c48df9ccc74a2b7695" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 33007 } }
I20260808 10:34:45.129662 3808 ts_tablet_manager.cc:1428] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:34:45.129703 3807 raft_consensus.cc:374] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:45.129758 3807 raft_consensus.cc:729] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a609b9615240c48df9ccc74a2b7695, State: Initialized, Role: FOLLOWER
I20260808 10:34:45.130013 3806 consensus_queue.cc:260] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 } } peers { permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 } } peers { permanent_uuid: "37a609b9615240c48df9ccc74a2b7695" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 33007 } }
I20260808 10:34:45.130234 3807 consensus_queue.cc:260] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 } } peers { permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 } } peers { permanent_uuid: "37a609b9615240c48df9ccc74a2b7695" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 33007 } }
I20260808 10:34:45.130724 3633 heartbeater.cc:503] Master 127.30.146.190:35393 was elected leader, sending a full tablet report...
I20260808 10:34:45.131109 3806 ts_tablet_manager.cc:1428] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:34:45.131304 3807 ts_tablet_manager.cc:1428] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:34:45.131680 3787 heartbeater.cc:503] Master 127.30.146.190:35393 was elected leader, sending a full tablet report...
I20260808 10:34:45.168905 3815 raft_consensus.cc:482] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:45.169255 3815 raft_consensus.cc:504] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 } } peers { permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 } } peers { permanent_uuid: "37a609b9615240c48df9ccc74a2b7695" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 33007 } }
I20260808 10:34:45.170593 3815 leader_election.cc:288] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c776a4e622b4151bdd5126b933e9ca0 (127.30.146.129:39891), 999e1cfd1f21408ea90cb491630dba66 (127.30.146.130:42603)
I20260808 10:34:45.174696 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.170980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:57965 (local address 127.30.146.129:39891)
0808 10:34:45.171105 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.171112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.171212 (+ 100us) server_negotiation.cc:407] Connection header received
0808 10:34:45.171327 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.171329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.171375 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.171476 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:45.171834 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.172346 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.173524 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.174088 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.174216 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.174271 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.174491 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.174577 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:34:45.174701 3789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.170871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:39891 (local address 127.30.146.131:57965)
0808 10:34:45.171078 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:45.171094 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:45.171215 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:45.171501 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:45.171504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:45.171523 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:45.171713 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.171723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.172468 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.172473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.173390 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.173401 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.174205 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.174209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.174326 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.174361 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 10:34:45.174457 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 10:34:45.174540 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 10:34:45.175272 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.170996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:41551 (local address 127.30.146.130:42603)
0808 10:34:45.171157 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.171161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.171460 (+ 299us) server_negotiation.cc:407] Connection header received
0808 10:34:45.171597 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.171600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.171681 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.171771 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:45.172083 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.172621 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.174031 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.174673 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.174841 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.174891 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.175087 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.175176 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 10:34:45.175498 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d7d10354d947db881ecf98a50357f2" candidate_uuid: "37a609b9615240c48df9ccc74a2b7695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c776a4e622b4151bdd5126b933e9ca0" is_pre_election: true
I20260808 10:34:45.175648 3816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.170918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:42603 (local address 127.30.146.131:41551)
0808 10:34:45.171336 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:45.171350 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:45.171479 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:45.171794 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:45.171798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:45.171816 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:45.171949 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.171955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.172746 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.172753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.173881 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:45.173893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.174846 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.174850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:45.174951 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.174972 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:34:45.175461 (+ 489us) client_negotiation.cc:240] Negotiation successful
0808 10:34:45.175538 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":141,"threads_started":1}
I20260808 10:34:45.175875 3441 raft_consensus.cc:2454] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37a609b9615240c48df9ccc74a2b7695 in term 0.
I20260808 10:34:45.176440 3583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d7d10354d947db881ecf98a50357f2" candidate_uuid: "37a609b9615240c48df9ccc74a2b7695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "999e1cfd1f21408ea90cb491630dba66" is_pre_election: true
I20260808 10:34:45.176489 3657 leader_election.cc:302] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37a609b9615240c48df9ccc74a2b7695, 4c776a4e622b4151bdd5126b933e9ca0; no voters:
I20260808 10:34:45.176728 3583 raft_consensus.cc:2454] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37a609b9615240c48df9ccc74a2b7695 in term 0.
I20260808 10:34:45.176833 3815 raft_consensus.cc:2781] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:45.176985 3815 raft_consensus.cc:482] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:45.177055 3815 raft_consensus.cc:3037] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:45.178284 3815 raft_consensus.cc:504] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 } } peers { permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 } } peers { permanent_uuid: "37a609b9615240c48df9ccc74a2b7695" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 33007 } }
I20260808 10:34:45.178715 3815 leader_election.cc:288] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [CANDIDATE]: Term 1 election: Requested vote from peers 4c776a4e622b4151bdd5126b933e9ca0 (127.30.146.129:39891), 999e1cfd1f21408ea90cb491630dba66 (127.30.146.130:42603)
I20260808 10:34:45.179041 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d7d10354d947db881ecf98a50357f2" candidate_uuid: "37a609b9615240c48df9ccc74a2b7695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c776a4e622b4151bdd5126b933e9ca0"
I20260808 10:34:45.179118 3583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d7d10354d947db881ecf98a50357f2" candidate_uuid: "37a609b9615240c48df9ccc74a2b7695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "999e1cfd1f21408ea90cb491630dba66"
I20260808 10:34:45.179198 3441 raft_consensus.cc:3037] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:45.179292 3583 raft_consensus.cc:3037] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:45.180367 3441 raft_consensus.cc:2454] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37a609b9615240c48df9ccc74a2b7695 in term 1.
I20260808 10:34:45.180450 3583 raft_consensus.cc:2454] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37a609b9615240c48df9ccc74a2b7695 in term 1.
I20260808 10:34:45.180745 3657 leader_election.cc:302] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37a609b9615240c48df9ccc74a2b7695, 4c776a4e622b4151bdd5126b933e9ca0; no voters:
I20260808 10:34:45.180964 3815 raft_consensus.cc:2781] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:45.181285 3815 raft_consensus.cc:686] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 1 LEADER]: Becoming Leader. State: Replica: 37a609b9615240c48df9ccc74a2b7695, State: Running, Role: LEADER
I20260808 10:34:45.181679 3815 consensus_queue.cc:237] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 } } peers { permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 } } peers { permanent_uuid: "37a609b9615240c48df9ccc74a2b7695" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 33007 } }
I20260808 10:34:45.185482 3284 catalog_manager.cc:5495] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37a609b9615240c48df9ccc74a2b7695 (127.30.146.131). New cstate: current_term: 1 leader_uuid: "37a609b9615240c48df9ccc74a2b7695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37a609b9615240c48df9ccc74a2b7695" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 33007 } health_report { overall_health: HEALTHY } } }
I20260808 10:34:45.202561 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.198817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.30.146.131:33007)
0808 10:34:45.198957 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.198961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.198975 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:34:45.199050 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.199052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.199108 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.199198 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:45.199499 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.200002 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.200997 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:45.201230 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:45.201310 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:45.201637 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:45.202106 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:45.202204 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.202312 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.202370 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:34:45.209069 3583 raft_consensus.cc:1261] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66 [term 1 FOLLOWER]: Refusing update from remote peer 37a609b9615240c48df9ccc74a2b7695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:45.209069 3441 raft_consensus.cc:1261] T d5d7d10354d947db881ecf98a50357f2 P 4c776a4e622b4151bdd5126b933e9ca0 [term 1 FOLLOWER]: Refusing update from remote peer 37a609b9615240c48df9ccc74a2b7695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:45.209954 3815 consensus_queue.cc:1035] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c776a4e622b4151bdd5126b933e9ca0" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 39891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:45.210249 3817 consensus_queue.cc:1035] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "999e1cfd1f21408ea90cb491630dba66" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:45.220557 3823 mvcc.cc:205] Tried to move back new op lower bound from 7316214928208674816 to 7316214928107409408. Current Snapshot: MvccSnapshot[applied={T|T < 7316214928208674816}]
I20260808 10:34:45.228199 3284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:45.228410 3284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:45.231331 3284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e7769eb0ba8457eb588a57efd526924: Sending DeleteTablet for 3 replicas of tablet d5d7d10354d947db881ecf98a50357f2
I20260808 10:34:45.232234 3561 tablet_service.cc:1514] Processing DeleteTablet for tablet d5d7d10354d947db881ecf98a50357f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:45 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:38594
I20260808 10:34:45.232223 3421 tablet_service.cc:1514] Processing DeleteTablet for tablet d5d7d10354d947db881ecf98a50357f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:45 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:45714
I20260808 10:34:45.232352 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet d5d7d10354d947db881ecf98a50357f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:45 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:49270
I20260808 10:34:45.233179 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 3356
I20260808 10:34:45.241122 3830 tablet_replica.cc:330] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66: stopping tablet replica
I20260808 10:34:45.241410 3831 tablet_replica.cc:330] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: stopping tablet replica
I20260808 10:34:45.241626 3830 raft_consensus.cc:2229] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:45.241860 3831 raft_consensus.cc:2229] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:34:45.242084 3830 raft_consensus.cc:2258] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:45.242340 3831 raft_consensus.cc:2258] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:45.243882 3830 ts_tablet_manager.cc:1905] T d5d7d10354d947db881ecf98a50357f2 P 999e1cfd1f21408ea90cb491630dba66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:45.244079 3831 ts_tablet_manager.cc:1905] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:34:45.245566 3269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:39891 (error 108)
I20260808 10:34:45.246474 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 3495
I20260808 10:34:45.249863 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.249434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:39891 (local address 127.0.0.1:45716)
0808 10:34:45.249668 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:45.249757 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:39891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260808 10:34:45.249948 3831 ts_tablet_manager.cc:1918] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:45.250051 3831 log.cc:1196] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/d5d7d10354d947db881ecf98a50357f2
W20260808 10:34:45.250229 3269 catalog_manager.cc:4584] TS 4c776a4e622b4151bdd5126b933e9ca0 (127.30.146.129:39891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5d7d10354d947db881ecf98a50357f2: Network error: Client connection negotiation failed: client connection to 127.30.146.129:39891: connect: Connection refused (error 111)
I20260808 10:34:45.250448 3831 ts_tablet_manager.cc:1939] T d5d7d10354d947db881ecf98a50357f2 P 37a609b9615240c48df9ccc74a2b7695: Deleting consensus metadata
I20260808 10:34:45.251744 3269 catalog_manager.cc:4841] TS 37a609b9615240c48df9ccc74a2b7695 (127.30.146.131:33007): tablet d5d7d10354d947db881ecf98a50357f2 (table test-table [id=fb570a60b0a34fc88c98e959dad19bc0]) successfully deleted
I20260808 10:34:45.255501 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 3638
I20260808 10:34:45.255554 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.255287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:42603 (local address 127.0.0.1:38604)
0808 10:34:45.255423 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:45.255484 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.130:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 10:34:45.263702 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 3250
2026-08-08T10:34:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2480 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T10:34:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:45Z Disabled control of system clock
I20260808 10:34:45.299886 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:46445
--webserver_interface=127.30.146.190
--webserver_port=0
--builtin_ntp_servers=127.30.146.148:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:46445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:34:45.418452 3839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:45.418731 3839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:45.418772 3839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:45.422891 3839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:45.422966 3839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:45.422981 3839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:45.422998 3839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:45.423055 3839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:45.428238 3839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:46445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:46445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:45.429528 3839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:45.430969 3839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:45.440105 3849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:45.440124 3844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:45.440325 3839 server_base.cc:1034] running on GCE node
W20260808 10:34:45.440095 3847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:45.441066 3839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:45.442407 3839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:45.443583 3839 hybrid_clock.cc:648] HybridClock initialized: now 1786185285443559 us; error 37 us; skew 500 ppm
I20260808 10:34:45.445639 3839 webserver.cc:460] Webserver started at http://127.30.146.190:39723/ using document root <none> and password file <none>
I20260808 10:34:45.446300 3839 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:45.446372 3839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:45.446605 3839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:45.448830 3839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "703133a8cc6f4927b643c8cc1bd3af86"
format_stamp: "Formatted at 2026-08-08 10:34:45 on dist-test-slave-9gcw"
server_key: "66f5ddd047bd944037ad967d849a82f5"
server_key_iv: "80357c490a2c83d386733a403ee1de61"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.449462 3839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "703133a8cc6f4927b643c8cc1bd3af86"
format_stamp: "Formatted at 2026-08-08 10:34:45 on dist-test-slave-9gcw"
server_key: "66f5ddd047bd944037ad967d849a82f5"
server_key_iv: "80357c490a2c83d386733a403ee1de61"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.453506 3839 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:34:45.456046 3855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:45.457412 3839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:34:45.457551 3839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "703133a8cc6f4927b643c8cc1bd3af86"
format_stamp: "Formatted at 2026-08-08 10:34:45 on dist-test-slave-9gcw"
server_key: "66f5ddd047bd944037ad967d849a82f5"
server_key_iv: "80357c490a2c83d386733a403ee1de61"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.457665 3839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:45.462436 3839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:45.463073 3839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:45.463267 3839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:45.471088 3839 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:46445
I20260808 10:34:45.471122 3912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:46445 every 8 connection(s)
I20260808 10:34:45.472143 3839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:45.475756 3913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:45.476606 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 3839
I20260808 10:34:45.476763 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:45.477106 31306 external_mini_cluster.cc:1428] Setting key 4cdff7fa6d97be6a1d87bc57aeb0a8df
I20260808 10:34:45.482378 3913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86: Bootstrap starting.
I20260808 10:34:45.483180 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.478635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36540 (local address 127.30.146.190:46445)
0808 10:34:45.479338 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.479355 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.479393 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:34:45.480088 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.480106 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.480513 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.480915 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:45.481765 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:45.481770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:45.481785 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:45.481804 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:45.481860 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:45.481877 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:45.482041 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.482117 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.482392 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":146,"threads_started":1}
I20260808 10:34:45.485100 3913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:45.486029 3913 log.cc:824] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:45.488044 3913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86: No bootstrap required, opened a new log
I20260808 10:34:45.492058 3913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703133a8cc6f4927b643c8cc1bd3af86" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46445 } }
I20260808 10:34:45.492334 3913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:45.492430 3913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 703133a8cc6f4927b643c8cc1bd3af86, State: Initialized, Role: FOLLOWER
I20260808 10:34:45.492933 3913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703133a8cc6f4927b643c8cc1bd3af86" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46445 } }
I20260808 10:34:45.493093 3913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:45.493209 3913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:45.493367 3913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:45.494390 3913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703133a8cc6f4927b643c8cc1bd3af86" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46445 } }
I20260808 10:34:45.494755 3913 leader_election.cc:302] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 703133a8cc6f4927b643c8cc1bd3af86; no voters:
I20260808 10:34:45.495103 3913 leader_election.cc:288] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:45.495275 3918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:45.495518 3918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [term 1 LEADER]: Becoming Leader. State: Replica: 703133a8cc6f4927b643c8cc1bd3af86, State: Running, Role: LEADER
I20260808 10:34:45.495888 3918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703133a8cc6f4927b643c8cc1bd3af86" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46445 } }
I20260808 10:34:45.496548 3913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:45.497727 3919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "703133a8cc6f4927b643c8cc1bd3af86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703133a8cc6f4927b643c8cc1bd3af86" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46445 } } }
I20260808 10:34:45.497861 3919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:45.498277 3928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:45.498255 3920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 703133a8cc6f4927b643c8cc1bd3af86. Latest consensus state: current_term: 1 leader_uuid: "703133a8cc6f4927b643c8cc1bd3af86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703133a8cc6f4927b643c8cc1bd3af86" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46445 } } }
I20260808 10:34:45.498414 3920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:45.501685 3928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:45.508071 3928 catalog_manager.cc:1347] Generated new cluster ID: d956578a6e764d3f8703a7a46c79f0c1
I20260808 10:34:45.508162 3928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:45.517540 3928 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:45.517886 3928 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:45.523715 3928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86: Generated new TSK 0
I20260808 10:34:45.524435 3928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:34:45.528075 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46445
--builtin_ntp_servers=127.30.146.148:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:34:45.659632 3940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:45.659891 3940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:45.659932 3940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:45.664310 3940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:45.664379 3940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:45.664470 3940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:45.669411 3940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:45.670553 3940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:45.672014 3940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:45.682217 3946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:45.682430 3940 server_base.cc:1034] running on GCE node
W20260808 10:34:45.682449 3948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:45.682294 3945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:45.683121 3940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:45.683784 3940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:45.684995 3940 hybrid_clock.cc:648] HybridClock initialized: now 1786185285684967 us; error 62 us; skew 500 ppm
I20260808 10:34:45.687107 3940 webserver.cc:460] Webserver started at http://127.30.146.129:42419/ using document root <none> and password file <none>
I20260808 10:34:45.687713 3940 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:45.687809 3940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:45.688050 3940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:45.690361 3940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "e2b0c59c5742404cac92b7efe348ba1f"
format_stamp: "Formatted at 2026-08-08 10:34:45 on dist-test-slave-9gcw"
server_key: "a5739b73885bb1d7b3459616a3b1c9f1"
server_key_iv: "e49d4dd06229b0220334716026d0562f"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.691018 3940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "e2b0c59c5742404cac92b7efe348ba1f"
format_stamp: "Formatted at 2026-08-08 10:34:45 on dist-test-slave-9gcw"
server_key: "a5739b73885bb1d7b3459616a3b1c9f1"
server_key_iv: "e49d4dd06229b0220334716026d0562f"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.695008 3940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:34:45.697921 3954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:45.699329 3940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:45.699473 3940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "e2b0c59c5742404cac92b7efe348ba1f"
format_stamp: "Formatted at 2026-08-08 10:34:45 on dist-test-slave-9gcw"
server_key: "a5739b73885bb1d7b3459616a3b1c9f1"
server_key_iv: "e49d4dd06229b0220334716026d0562f"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.699596 3940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:45.704340 3940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:45.705016 3940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:45.705265 3940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:45.705941 3940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:45.706916 3940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:45.706992 3940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:45.707060 3940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:45.707111 3940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:45.716471 3940 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:34939
I20260808 10:34:45.716495 4077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:34939 every 8 connection(s)
I20260808 10:34:45.717504 3940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:45.723918 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.719505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:54693 (local address 127.30.146.190:46445)
0808 10:34:45.719654 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.719658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.720772 (+ 1114us) server_negotiation.cc:407] Connection header received
0808 10:34:45.722248 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.722250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.722290 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.722374 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:45.723529 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:45.723531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:45.723534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:45.723543 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:45.723560 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:45.723566 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:45.723639 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.723765 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.723803 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:34:45.724771 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.719928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46445 (local address 127.30.146.129:54693)
0808 10:34:45.720605 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:45.720645 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:45.722036 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:45.722540 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:45.722546 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:45.722635 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:45.723282 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:34:45.723283 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:45.723325 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:34:45.723394 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:45.723641 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:45.723648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:45.723653 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:34:45.723784 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 10:34:45.723973 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":125,"threads_started":1}
I20260808 10:34:45.726020 4078 heartbeater.cc:347] Connected to a master server at 127.30.146.190:46445
I20260808 10:34:45.726269 4078 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:45.726480 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 3940
I20260808 10:34:45.726615 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:45.726768 4078 heartbeater.cc:511] Master 127.30.146.190:46445 requested a full tablet report, sending...
I20260808 10:34:45.726984 31306 external_mini_cluster.cc:1428] Setting key 8f59b159a2719bfd996fbc3c899be3db
I20260808 10:34:45.728497 3873 ts_manager.cc:194] Registered new tserver with Master: e2b0c59c5742404cac92b7efe348ba1f (127.30.146.129:34939)
I20260808 10:34:45.729128 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46445
--builtin_ntp_servers=127.30.146.148:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:34:45.846298 4082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:45.846614 4082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:45.846696 4082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:45.850847 4082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:45.850935 4082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:45.851025 4082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:45.856003 4082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:45.857276 4082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:45.858765 4082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:45.868815 4090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:45.868798 4089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:45.868794 4092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:45.869287 4082 server_base.cc:1034] running on GCE node
I20260808 10:34:45.869692 4082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:45.870342 4082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:45.871507 4082 hybrid_clock.cc:648] HybridClock initialized: now 1786185285871476 us; error 43 us; skew 500 ppm
I20260808 10:34:45.873548 4082 webserver.cc:460] Webserver started at http://127.30.146.130:41951/ using document root <none> and password file <none>
I20260808 10:34:45.874155 4082 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:45.874221 4082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:45.874451 4082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:45.876657 4082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "d56e2a0df19c43dfa0fda77d5c5d997a"
format_stamp: "Formatted at 2026-08-08 10:34:45 on dist-test-slave-9gcw"
server_key: "1dd30e3609f863a13265d38f886641b4"
server_key_iv: "565347e7eab3048c2ed9f984b5d459e7"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.877351 4082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "d56e2a0df19c43dfa0fda77d5c5d997a"
format_stamp: "Formatted at 2026-08-08 10:34:45 on dist-test-slave-9gcw"
server_key: "1dd30e3609f863a13265d38f886641b4"
server_key_iv: "565347e7eab3048c2ed9f984b5d459e7"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.881685 4082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:45.884395 4098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:45.885805 4082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:45.885950 4082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "d56e2a0df19c43dfa0fda77d5c5d997a"
format_stamp: "Formatted at 2026-08-08 10:34:45 on dist-test-slave-9gcw"
server_key: "1dd30e3609f863a13265d38f886641b4"
server_key_iv: "565347e7eab3048c2ed9f984b5d459e7"
server_key_version: "encryptionkey@0"
I20260808 10:34:45.886076 4082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:45.891372 4082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:45.892122 4082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:45.892314 4082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:45.892881 4082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:45.893862 4082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:45.893935 4082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:45.894002 4082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:45.894057 4082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:45.904094 4082 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:46835
I20260808 10:34:45.904160 4221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:46835 every 8 connection(s)
I20260808 10:34:45.905108 4082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:45.906543 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 4082
I20260808 10:34:45.906667 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:45.906984 31306 external_mini_cluster.cc:1428] Setting key 37f9241c23d2498b184ff9a5a24c6b9e
I20260808 10:34:45.909660 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46445
--builtin_ntp_servers=127.30.146.148:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 10:34:45.912249 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.908891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:37317 (local address 127.30.146.190:46445)
0808 10:34:45.909044 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:45.909049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:45.909066 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:34:45.910342 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:45.910348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:45.910394 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:45.910491 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:45.911759 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:45.911801 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:45.911803 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:45.911814 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:45.911835 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:45.911843 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:45.911920 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:45.912100 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 10:34:45.912139 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:34:45.913553 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:45.907771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46445 (local address 127.30.146.130:37317)
0808 10:34:45.908577 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:45.908639 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:45.910128 (+ 1489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:45.910656 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:45.910663 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:45.910753 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:45.911394 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:34:45.911417 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:45.911482 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:34:45.911589 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:45.911937 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:45.911948 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:45.911957 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:34:45.912150 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 10:34:45.912447 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":142,"threads_started":1}
I20260808 10:34:45.914788 4222 heartbeater.cc:347] Connected to a master server at 127.30.146.190:46445
I20260808 10:34:45.915069 4222 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:45.915634 4222 heartbeater.cc:511] Master 127.30.146.190:46445 requested a full tablet report, sending...
I20260808 10:34:45.916886 3873 ts_manager.cc:194] Registered new tserver with Master: d56e2a0df19c43dfa0fda77d5c5d997a (127.30.146.130:46835)
W20260808 10:34:46.028760 4226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:46.029089 4226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:46.029197 4226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:46.033402 4226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:46.033494 4226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:46.033589 4226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:46.038590 4226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:46445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:46.039876 4226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:46.041489 4226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:46.051517 4232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:46.051527 4234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:46.051558 4231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:46.051563 4226 server_base.cc:1034] running on GCE node
I20260808 10:34:46.052222 4226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:46.052881 4226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:46.054041 4226 hybrid_clock.cc:648] HybridClock initialized: now 1786185286054003 us; error 53 us; skew 500 ppm
I20260808 10:34:46.055982 4226 webserver.cc:460] Webserver started at http://127.30.146.131:40033/ using document root <none> and password file <none>
I20260808 10:34:46.056578 4226 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:46.056645 4226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:46.056926 4226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:46.059219 4226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "ef05f39a8c144d1386ee09282adb7c25"
format_stamp: "Formatted at 2026-08-08 10:34:46 on dist-test-slave-9gcw"
server_key: "6babe1104ce412ca89643eca53939cec"
server_key_iv: "1d012926d83c759261b90b04348edced"
server_key_version: "encryptionkey@0"
I20260808 10:34:46.059862 4226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "ef05f39a8c144d1386ee09282adb7c25"
format_stamp: "Formatted at 2026-08-08 10:34:46 on dist-test-slave-9gcw"
server_key: "6babe1104ce412ca89643eca53939cec"
server_key_iv: "1d012926d83c759261b90b04348edced"
server_key_version: "encryptionkey@0"
I20260808 10:34:46.064101 4226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:34:46.066807 4241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:46.068104 4226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:34:46.068233 4226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "ef05f39a8c144d1386ee09282adb7c25"
format_stamp: "Formatted at 2026-08-08 10:34:46 on dist-test-slave-9gcw"
server_key: "6babe1104ce412ca89643eca53939cec"
server_key_iv: "1d012926d83c759261b90b04348edced"
server_key_version: "encryptionkey@0"
I20260808 10:34:46.068352 4226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:46.073110 4226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:46.073854 4226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:46.074044 4226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:46.074621 4226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:46.075583 4226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:46.075630 4226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:46.075700 4226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:46.075742 4226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:46.084894 4226 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:38783
I20260808 10:34:46.084918 4361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:38783 every 8 connection(s)
I20260808 10:34:46.085896 4226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:46.088254 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 4226
I20260808 10:34:46.088393 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:46.088680 31306 external_mini_cluster.cc:1428] Setting key 4181cb3a66ce38e0a34e14e079b9b6c6
I20260808 10:34:46.092720 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.087843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:42777 (local address 127.30.146.190:46445)
0808 10:34:46.088067 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:46.088074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:46.089236 (+ 1162us) server_negotiation.cc:407] Connection header received
0808 10:34:46.090931 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:46.090934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:46.090980 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:46.091068 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:46.092314 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:46.092317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:46.092319 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:46.092331 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:46.092351 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:46.092359 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:46.092436 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:46.092564 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 10:34:46.092602 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:34:46.093705 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.088220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46445 (local address 127.30.146.131:42777)
0808 10:34:46.089035 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:46.089089 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:46.090692 (+ 1603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:46.091268 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:46.091276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:46.091409 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:46.092052 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:34:46.092054 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:46.092097 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:34:46.092166 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:46.092441 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:46.092448 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:46.092452 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:34:46.092587 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 10:34:46.092784 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":177,"threads_started":1}
I20260808 10:34:46.094779 4362 heartbeater.cc:347] Connected to a master server at 127.30.146.190:46445
I20260808 10:34:46.095036 4362 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:46.095479 4362 heartbeater.cc:511] Master 127.30.146.190:46445 requested a full tablet report, sending...
I20260808 10:34:46.096488 3873 ts_manager.cc:194] Registered new tserver with Master: ef05f39a8c144d1386ee09282adb7c25 (127.30.146.131:38783)
I20260808 10:34:46.103323 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:34:46.107425 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.105532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36554 (local address 127.30.146.190:46445)
0808 10:34:46.105711 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:46.105717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:46.105856 (+ 139us) server_negotiation.cc:407] Connection header received
0808 10:34:46.106021 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:46.106024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:46.106061 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:46.106154 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:46.107007 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:46.107011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:46.107013 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:46.107024 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:46.107045 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:46.107054 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:46.107153 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:46.107236 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 10:34:46.107281 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:34:46.110385 3873 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:34:46.113965 3873 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:46.126801 4374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.122925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34939 (local address 127.0.0.1:38454)
0808 10:34:46.123904 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:46.123961 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:46.124070 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:46.124847 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:46.124854 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:46.124932 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:46.126042 (+ 1110us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:34:46.126044 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:46.126056 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:34:46.126089 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:46.126484 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:46.126488 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:46.126490 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:34:46.126556 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:34:46.126641 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":95,"threads_started":1}
I20260808 10:34:46.126817 4377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.122943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:38783 (local address 127.0.0.1:45916)
0808 10:34:46.123902 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:46.123957 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:46.124053 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:46.125094 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:46.125099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:46.125117 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:46.125722 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:34:46.125726 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:46.125750 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:34:46.125829 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:46.126483 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:46.126489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:46.126493 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:34:46.126604 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:34:46.126656 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":112,"threads_started":1}
I20260808 10:34:46.127351 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.123216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:46835 (local address 127.0.0.1:44684)
0808 10:34:46.124269 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:46.124283 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:46.124360 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:46.125579 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:46.125582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:46.125594 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:46.126000 (+ 406us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:34:46.126002 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:46.126010 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:34:46.126029 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:46.126443 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:46.126456 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:46.126464 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:34:46.126567 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:34:46.127233 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":592,"thread_start_us":94,"threads_started":1}
I20260808 10:34:46.127420 4376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.123115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.30.146.131:38783)
0808 10:34:46.124637 (+ 1522us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:46.124647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:46.124668 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:34:46.124714 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:46.124718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:46.124923 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:46.125376 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:46.126157 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:46.126166 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:46.126180 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:46.126195 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:46.126239 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:46.126268 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:46.126416 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:46.127198 (+ 782us) server_negotiation.cc:299] Negotiation successful
0808 10:34:46.127273 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1400,"thread_start_us":164,"threads_started":1}
I20260808 10:34:46.127552 4379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.123833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38454 (local address 127.30.146.129:34939)
0808 10:34:46.124328 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:46.124340 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:46.124373 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:34:46.124451 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:46.124458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:46.124666 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:46.124819 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:46.126223 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:46.126230 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:46.126239 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:46.126251 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:46.126331 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:46.126367 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:46.127237 (+ 870us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:46.127289 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 10:34:46.127375 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":121,"threads_started":1}
I20260808 10:34:46.127609 4380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.124250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.30.146.130:46835)
0808 10:34:46.124591 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:46.124601 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:46.124622 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:34:46.124670 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:46.124675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:46.124828 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:46.124948 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:46.126154 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:46.126163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:46.126178 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:46.126192 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:46.126240 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:46.126280 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:46.126392 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:46.126681 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 10:34:46.126757 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":117,"threads_started":1}
I20260808 10:34:46.130427 4293 tablet_service.cc:1467] Processing CreateTablet for tablet 6283d9bc9da242f5b2e768ed18440bde (DEFAULT_TABLE table=test-table [id=6374b94561654c8590af24d45c860f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:46.130429 4147 tablet_service.cc:1467] Processing CreateTablet for tablet 6283d9bc9da242f5b2e768ed18440bde (DEFAULT_TABLE table=test-table [id=6374b94561654c8590af24d45c860f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:46.130429 4004 tablet_service.cc:1467] Processing CreateTablet for tablet 6283d9bc9da242f5b2e768ed18440bde (DEFAULT_TABLE table=test-table [id=6374b94561654c8590af24d45c860f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:46.131621 4004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6283d9bc9da242f5b2e768ed18440bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:46.131621 4147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6283d9bc9da242f5b2e768ed18440bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:46.131649 4293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6283d9bc9da242f5b2e768ed18440bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:46.139612 4382 tablet_bootstrap.cc:492] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f: Bootstrap starting.
I20260808 10:34:46.140156 4381 tablet_bootstrap.cc:492] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: Bootstrap starting.
I20260808 10:34:46.140268 4383 tablet_bootstrap.cc:492] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: Bootstrap starting.
I20260808 10:34:46.142136 4381 tablet_bootstrap.cc:654] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:46.142227 4382 tablet_bootstrap.cc:654] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:46.142300 4383 tablet_bootstrap.cc:654] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:46.143009 4381 log.cc:824] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:46.143129 4382 log.cc:824] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:46.143129 4383 log.cc:824] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:46.145015 4382 tablet_bootstrap.cc:492] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f: No bootstrap required, opened a new log
I20260808 10:34:46.145100 4383 tablet_bootstrap.cc:492] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: No bootstrap required, opened a new log
I20260808 10:34:46.145100 4381 tablet_bootstrap.cc:492] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: No bootstrap required, opened a new log
I20260808 10:34:46.145272 4382 ts_tablet_manager.cc:1397] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:34:46.145303 4383 ts_tablet_manager.cc:1397] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:34:46.145303 4381 ts_tablet_manager.cc:1397] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260808 10:34:46.149003 4382 raft_consensus.cc:348] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 } } peers { permanent_uuid: "e2b0c59c5742404cac92b7efe348ba1f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34939 } } peers { permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 } }
I20260808 10:34:46.149349 4382 raft_consensus.cc:374] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:46.149408 4382 raft_consensus.cc:729] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b0c59c5742404cac92b7efe348ba1f, State: Initialized, Role: FOLLOWER
I20260808 10:34:46.149896 4382 consensus_queue.cc:260] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 } } peers { permanent_uuid: "e2b0c59c5742404cac92b7efe348ba1f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34939 } } peers { permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 } }
I20260808 10:34:46.150341 4381 raft_consensus.cc:348] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 } } peers { permanent_uuid: "e2b0c59c5742404cac92b7efe348ba1f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34939 } } peers { permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 } }
I20260808 10:34:46.150635 4381 raft_consensus.cc:374] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:46.150635 4078 heartbeater.cc:503] Master 127.30.146.190:46445 was elected leader, sending a full tablet report...
I20260808 10:34:46.150705 4381 raft_consensus.cc:729] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef05f39a8c144d1386ee09282adb7c25, State: Initialized, Role: FOLLOWER
I20260808 10:34:46.151072 4382 ts_tablet_manager.cc:1428] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:34:46.151142 4381 consensus_queue.cc:260] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 } } peers { permanent_uuid: "e2b0c59c5742404cac92b7efe348ba1f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34939 } } peers { permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 } }
I20260808 10:34:46.150341 4383 raft_consensus.cc:348] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 } } peers { permanent_uuid: "e2b0c59c5742404cac92b7efe348ba1f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34939 } } peers { permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 } }
I20260808 10:34:46.151844 4362 heartbeater.cc:503] Master 127.30.146.190:46445 was elected leader, sending a full tablet report...
I20260808 10:34:46.151877 4383 raft_consensus.cc:374] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:46.151940 4383 raft_consensus.cc:729] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d56e2a0df19c43dfa0fda77d5c5d997a, State: Initialized, Role: FOLLOWER
I20260808 10:34:46.152218 4381 ts_tablet_manager.cc:1428] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260808 10:34:46.152411 4383 consensus_queue.cc:260] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 } } peers { permanent_uuid: "e2b0c59c5742404cac92b7efe348ba1f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34939 } } peers { permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 } }
I20260808 10:34:46.153452 4383 ts_tablet_manager.cc:1428] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 10:34:46.153795 4222 heartbeater.cc:503] Master 127.30.146.190:46445 was elected leader, sending a full tablet report...
W20260808 10:34:46.156188 4223 tablet.cc:2369] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:34:46.218853 4079 tablet.cc:2369] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:34:46.336946 4363 tablet.cc:2369] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:34:46.522320 4387 raft_consensus.cc:482] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:46.522704 4387 raft_consensus.cc:504] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 } } peers { permanent_uuid: "e2b0c59c5742404cac92b7efe348ba1f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34939 } } peers { permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 } }
I20260808 10:34:46.524015 4387 leader_election.cc:288] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef05f39a8c144d1386ee09282adb7c25 (127.30.146.131:38783), d56e2a0df19c43dfa0fda77d5c5d997a (127.30.146.130:46835)
I20260808 10:34:46.526340 4391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.524302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:38783 (local address 127.30.146.129:57657)
0808 10:34:46.524662 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:46.524683 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:46.524806 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:46.525054 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:46.525057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:46.525072 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:46.525667 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:34:46.525670 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:46.525684 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:34:46.525723 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:46.526018 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:46.526022 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:46.526025 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:34:46.526129 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 10:34:46.526171 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":80,"threads_started":1}
I20260808 10:34:46.526360 4376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.524465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:57657 (local address 127.30.146.131:38783)
0808 10:34:46.524604 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:46.524609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:46.524770 (+ 161us) server_negotiation.cc:407] Connection header received
0808 10:34:46.524894 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:46.524897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:46.524934 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:46.525031 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:46.525866 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:46.525870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:46.525872 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:46.525886 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:46.525907 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:46.525917 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:46.526013 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:46.526150 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 10:34:46.526197 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:34:46.526425 4380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.524445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:52941 (local address 127.30.146.130:46835)
0808 10:34:46.524593 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:46.524598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:46.524794 (+ 196us) server_negotiation.cc:407] Connection header received
0808 10:34:46.524952 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:46.524956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:46.525000 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:46.525088 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:46.526019 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:46.526023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:46.526026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:46.526039 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:46.526064 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:46.526079 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:46.526166 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:46.526248 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 10:34:46.526284 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:34:46.526465 4390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.524318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:46835 (local address 127.30.146.129:52941)
0808 10:34:46.524703 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:46.524715 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:46.524834 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:46.525111 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:46.525114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:46.525129 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:46.525860 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:34:46.525863 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:46.525873 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:34:46.525901 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:46.526165 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:46.526168 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:46.526172 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:34:46.526254 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:34:46.526286 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":96,"threads_started":1}
I20260808 10:34:46.527002 4314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6283d9bc9da242f5b2e768ed18440bde" candidate_uuid: "e2b0c59c5742404cac92b7efe348ba1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef05f39a8c144d1386ee09282adb7c25" is_pre_election: true
I20260808 10:34:46.527204 4170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6283d9bc9da242f5b2e768ed18440bde" candidate_uuid: "e2b0c59c5742404cac92b7efe348ba1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" is_pre_election: true
I20260808 10:34:46.527299 4314 raft_consensus.cc:2454] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b0c59c5742404cac92b7efe348ba1f in term 0.
I20260808 10:34:46.527499 4170 raft_consensus.cc:2454] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b0c59c5742404cac92b7efe348ba1f in term 0.
I20260808 10:34:46.527820 3956 leader_election.cc:302] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e2b0c59c5742404cac92b7efe348ba1f, ef05f39a8c144d1386ee09282adb7c25; no voters:
I20260808 10:34:46.528115 4387 raft_consensus.cc:2781] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:46.528190 4387 raft_consensus.cc:482] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:46.528221 4387 raft_consensus.cc:3037] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:46.529439 4387 raft_consensus.cc:504] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 } } peers { permanent_uuid: "e2b0c59c5742404cac92b7efe348ba1f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34939 } } peers { permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 } }
I20260808 10:34:46.529812 4387 leader_election.cc:288] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [CANDIDATE]: Term 1 election: Requested vote from peers ef05f39a8c144d1386ee09282adb7c25 (127.30.146.131:38783), d56e2a0df19c43dfa0fda77d5c5d997a (127.30.146.130:46835)
I20260808 10:34:46.530143 4170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6283d9bc9da242f5b2e768ed18440bde" candidate_uuid: "e2b0c59c5742404cac92b7efe348ba1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a"
I20260808 10:34:46.530155 4314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6283d9bc9da242f5b2e768ed18440bde" candidate_uuid: "e2b0c59c5742404cac92b7efe348ba1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef05f39a8c144d1386ee09282adb7c25"
I20260808 10:34:46.530259 4170 raft_consensus.cc:3037] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:46.530285 4314 raft_consensus.cc:3037] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:46.531356 4170 raft_consensus.cc:2454] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b0c59c5742404cac92b7efe348ba1f in term 1.
I20260808 10:34:46.531450 4314 raft_consensus.cc:2454] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b0c59c5742404cac92b7efe348ba1f in term 1.
I20260808 10:34:46.531638 3958 leader_election.cc:302] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d56e2a0df19c43dfa0fda77d5c5d997a, e2b0c59c5742404cac92b7efe348ba1f; no voters:
I20260808 10:34:46.531787 4387 raft_consensus.cc:2781] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:46.531984 4387 raft_consensus.cc:686] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [term 1 LEADER]: Becoming Leader. State: Replica: e2b0c59c5742404cac92b7efe348ba1f, State: Running, Role: LEADER
I20260808 10:34:46.532351 4387 consensus_queue.cc:237] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 } } peers { permanent_uuid: "e2b0c59c5742404cac92b7efe348ba1f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34939 } } peers { permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 } }
I20260808 10:34:46.535689 3872 catalog_manager.cc:5495] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b0c59c5742404cac92b7efe348ba1f (127.30.146.129). New cstate: current_term: 1 leader_uuid: "e2b0c59c5742404cac92b7efe348ba1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b0c59c5742404cac92b7efe348ba1f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:46.581709 4379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.579853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38462 (local address 127.30.146.129:34939)
0808 10:34:46.580002 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:46.580006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:46.580021 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:34:46.580135 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:46.580138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:46.580196 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:46.580265 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:46.581003 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:46.581006 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:46.581009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:46.581031 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:46.581050 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:46.581058 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:46.581132 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:46.581289 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 10:34:46.581403 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:34:46.588070 4314 raft_consensus.cc:1261] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25 [term 1 FOLLOWER]: Refusing update from remote peer e2b0c59c5742404cac92b7efe348ba1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:46.588140 4170 raft_consensus.cc:1261] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a [term 1 FOLLOWER]: Refusing update from remote peer e2b0c59c5742404cac92b7efe348ba1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:46.588845 4392 consensus_queue.cc:1035] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef05f39a8c144d1386ee09282adb7c25" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:46.589068 4387 consensus_queue.cc:1035] T 6283d9bc9da242f5b2e768ed18440bde P e2b0c59c5742404cac92b7efe348ba1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d56e2a0df19c43dfa0fda77d5c5d997a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:46.595829 4397 mvcc.cc:205] Tried to move back new op lower bound from 7316214933857447936 to 7316214933639188480. Current Snapshot: MvccSnapshot[applied={T|T < 7316214933857447936}]
I20260808 10:34:46.597357 4399 mvcc.cc:205] Tried to move back new op lower bound from 7316214933857447936 to 7316214933639188480. Current Snapshot: MvccSnapshot[applied={T|T < 7316214933857447936}]
I20260808 10:34:46.606058 3872 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:46.606251 3872 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:46.608767 3872 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 703133a8cc6f4927b643c8cc1bd3af86: Sending DeleteTablet for 3 replicas of tablet 6283d9bc9da242f5b2e768ed18440bde
I20260808 10:34:46.609593 4004 tablet_service.cc:1514] Processing DeleteTablet for tablet 6283d9bc9da242f5b2e768ed18440bde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:46 UTC) from {username='slave'} at 127.0.0.1:38454
I20260808 10:34:46.609597 4147 tablet_service.cc:1514] Processing DeleteTablet for tablet 6283d9bc9da242f5b2e768ed18440bde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:46 UTC) from {username='slave'} at 127.0.0.1:44684
I20260808 10:34:46.609640 4293 tablet_service.cc:1514] Processing DeleteTablet for tablet 6283d9bc9da242f5b2e768ed18440bde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:46 UTC) from {username='slave'} at 127.0.0.1:45916
I20260808 10:34:46.610146 4409 tablet_replica.cc:330] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: stopping tablet replica
I20260808 10:34:46.610412 4407 tablet_replica.cc:330] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: stopping tablet replica
I20260808 10:34:46.610502 4409 raft_consensus.cc:2229] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:46.610675 4407 raft_consensus.cc:2229] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:46.610832 4409 raft_consensus.cc:2258] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:46.610945 4407 raft_consensus.cc:2258] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:46.611214 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 3940
I20260808 10:34:46.612841 4409 ts_tablet_manager.cc:1905] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:46.614019 4407 ts_tablet_manager.cc:1905] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:46.615928 4409 ts_tablet_manager.cc:1918] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:46.616036 4409 log.cc:1196] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/6283d9bc9da242f5b2e768ed18440bde
I20260808 10:34:46.616356 4409 ts_tablet_manager.cc:1939] T 6283d9bc9da242f5b2e768ed18440bde P ef05f39a8c144d1386ee09282adb7c25: Deleting consensus metadata
I20260808 10:34:46.617105 4407 ts_tablet_manager.cc:1918] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:46.617252 4407 log.cc:1196] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/6283d9bc9da242f5b2e768ed18440bde
I20260808 10:34:46.617554 4407 ts_tablet_manager.cc:1939] T 6283d9bc9da242f5b2e768ed18440bde P d56e2a0df19c43dfa0fda77d5c5d997a: Deleting consensus metadata
I20260808 10:34:46.618312 3857 catalog_manager.cc:4841] TS ef05f39a8c144d1386ee09282adb7c25 (127.30.146.131:38783): tablet 6283d9bc9da242f5b2e768ed18440bde (table test-table [id=6374b94561654c8590af24d45c860f5c]) successfully deleted
I20260808 10:34:46.618772 3859 catalog_manager.cc:4841] TS d56e2a0df19c43dfa0fda77d5c5d997a (127.30.146.130:46835): tablet 6283d9bc9da242f5b2e768ed18440bde (table test-table [id=6374b94561654c8590af24d45c860f5c]) successfully deleted
W20260808 10:34:46.623353 3858 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:34939 (error 108)
I20260808 10:34:46.623654 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 4082
I20260808 10:34:46.625331 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.624173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34939 (local address 127.0.0.1:38466)
0808 10:34:46.625091 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:46.625200 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":810}
W20260808 10:34:46.625785 3858 catalog_manager.cc:4584] TS e2b0c59c5742404cac92b7efe348ba1f (127.30.146.129:34939): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6283d9bc9da242f5b2e768ed18440bde: Network error: Client connection negotiation failed: client connection to 127.30.146.129:34939: connect: Connection refused (error 111)
I20260808 10:34:46.631371 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 4226
I20260808 10:34:46.641322 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 3839
2026-08-08T10:34:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1375 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 10:34:46.665786 31306 mini_oidc.cc:152] Starting JWKS server
I20260808 10:34:46.667644 31306 webserver.cc:460] Webserver started at https://127.0.0.1:41755/ using document root <none> and password file <none>
I20260808 10:34:46.667845 31306 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:34:46.668133 31306 webserver.cc:460] Webserver started at http://127.0.0.1:38693/ using document root <none> and password file <none>
2026-08-08T10:34:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:46Z Disabled control of system clock
I20260808 10:34:46.678952 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:41961
--webserver_interface=127.30.146.190
--webserver_port=0
--builtin_ntp_servers=127.30.146.148:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:41961
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41755/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/testchainca.cert with env {}
W20260808 10:34:46.797606 4416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:46.797900 4416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:46.797941 4416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:46.801996 4416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:46.802060 4416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:46.802075 4416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:46.802093 4416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:46.802107 4416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:34:46.802124 4416 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:34:46.802137 4416 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41755/jwks.json
I20260808 10:34:46.807646 4416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:41961
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:41961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41755/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:46.808732 4416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:46.809733 4416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:46.818552 4421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:46.818632 4416 server_base.cc:1034] running on GCE node
W20260808 10:34:46.818715 4422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:46.818766 4424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:46.819443 4416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:46.820618 4416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:46.821815 4416 hybrid_clock.cc:648] HybridClock initialized: now 1786185286821784 us; error 48 us; skew 500 ppm
I20260808 10:34:46.823830 4416 webserver.cc:460] Webserver started at http://127.30.146.190:43257/ using document root <none> and password file <none>
I20260808 10:34:46.824433 4416 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:46.824532 4416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:46.824765 4416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:46.827019 4416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "9bbcacaf40474134b03c20dd3369405f"
format_stamp: "Formatted at 2026-08-08 10:34:46 on dist-test-slave-9gcw"
server_key: "9a6f884decf6506ad997be8f237bb341"
server_key_iv: "a36bf84d4e07eef5a41d1cafdf2d55e6"
server_key_version: "encryptionkey@0"
I20260808 10:34:46.827669 4416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "9bbcacaf40474134b03c20dd3369405f"
format_stamp: "Formatted at 2026-08-08 10:34:46 on dist-test-slave-9gcw"
server_key: "9a6f884decf6506ad997be8f237bb341"
server_key_iv: "a36bf84d4e07eef5a41d1cafdf2d55e6"
server_key_version: "encryptionkey@0"
I20260808 10:34:46.831748 4416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260808 10:34:46.834326 4431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:46.835796 4416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:34:46.835942 4416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "9bbcacaf40474134b03c20dd3369405f"
format_stamp: "Formatted at 2026-08-08 10:34:46 on dist-test-slave-9gcw"
server_key: "9a6f884decf6506ad997be8f237bb341"
server_key_iv: "a36bf84d4e07eef5a41d1cafdf2d55e6"
server_key_version: "encryptionkey@0"
I20260808 10:34:46.836057 4416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:46.849630 4416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:46.850399 4416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:46.850608 4416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:46.859367 4416 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:41961
I20260808 10:34:46.859382 4489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:41961 every 8 connection(s)
I20260808 10:34:46.860415 4416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:46.864097 4490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:46.866112 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 4416
I20260808 10:34:46.866291 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:46.866590 31306 external_mini_cluster.cc:1428] Setting key b045a267c6dc7a40f3bd94a50951996b
I20260808 10:34:46.872247 4490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f: Bootstrap starting.
I20260808 10:34:46.875185 4490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:46.876048 4490 log.cc:824] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:46.878247 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.868040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54476 (local address 127.30.146.190:41961)
0808 10:34:46.868827 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:46.868852 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:46.868901 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:34:46.869927 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:46.869955 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:46.870656 (+ 701us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:46.871070 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:46.871546 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:46.872942 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:46.874030 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:46.874695 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:46.874861 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:46.875979 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:46.875985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:46.876001 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:46.876537 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:46.876595 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:46.876617 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:46.876695 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:46.876950 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 10:34:46.877307 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":133,"threads_started":1}
I20260808 10:34:46.878525 4490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f: No bootstrap required, opened a new log
I20260808 10:34:46.881493 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:34:46.882997 4490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbcacaf40474134b03c20dd3369405f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41961 } }
I20260808 10:34:46.883237 4490 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:46.883308 4490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bbcacaf40474134b03c20dd3369405f, State: Initialized, Role: FOLLOWER
I20260808 10:34:46.883813 4490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbcacaf40474134b03c20dd3369405f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41961 } }
I20260808 10:34:46.883947 4490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:46.884045 4490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:46.884189 4490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:46.885245 4490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbcacaf40474134b03c20dd3369405f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41961 } }
I20260808 10:34:46.885592 4490 leader_election.cc:302] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bbcacaf40474134b03c20dd3369405f; no voters:
I20260808 10:34:46.885922 4490 leader_election.cc:288] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:46.886144 4495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:46.886438 4495 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [term 1 LEADER]: Becoming Leader. State: Replica: 9bbcacaf40474134b03c20dd3369405f, State: Running, Role: LEADER
I20260808 10:34:46.886801 4495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbcacaf40474134b03c20dd3369405f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41961 } }
I20260808 10:34:46.887276 4490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:46.889074 4496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bbcacaf40474134b03c20dd3369405f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbcacaf40474134b03c20dd3369405f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41961 } } }
I20260808 10:34:46.889259 4496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:46.889863 4497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bbcacaf40474134b03c20dd3369405f. Latest consensus state: current_term: 1 leader_uuid: "9bbcacaf40474134b03c20dd3369405f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbcacaf40474134b03c20dd3369405f" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41961 } } }
I20260808 10:34:46.890043 4497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f [sys.catalog]: This master's current role is: LEADER
W20260808 10:34:46.893086 4513 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:34:46.893251 4513 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:34:46.893328 4512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:46.894297 4512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:46.899894 4512 catalog_manager.cc:1347] Generated new cluster ID: b22a2576efb4423fa20392a381769737
I20260808 10:34:46.899988 4512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:46.927170 4512 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:46.928845 4512 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:46.935169 4512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bbcacaf40474134b03c20dd3369405f: Generated new TSK 0
I20260808 10:34:46.936054 4512 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 10:34:46.984413 4523 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 10:34:46.984587 4523 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 10:34:46.985112 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:46.958570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54486 (local address 127.30.146.190:41961)
0808 10:34:46.958819 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:46.958825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:46.959090 (+ 265us) server_negotiation.cc:407] Connection header received
0808 10:34:46.959361 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:46.959364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:46.959420 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:46.959524 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:34:46.959841 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:46.960888 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:46.962284 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:46.962536 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:46.962629 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:46.963097 (+ 468us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:34:46.984617 (+ 21520us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 10:34:46.984988 (+ 371us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54486: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":149}
W20260808 10:34:46.985263 4493 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54486: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 10:34:46.986831 4522 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:34:46.958418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:41961 (local address 127.0.0.1:54486)
0808 10:34:46.958977 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:46.958993 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:46.959159 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:46.959557 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:46.959560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:46.959569 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:34:46.959698 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:46.959705 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:46.961016 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:46.961019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:46.962140 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:46.962152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:46.962672 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:46.962676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:46.962779 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:46.962841 (+ 62us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:34:46.984958 (+ 22117us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 10:34:46.985252 (+ 294us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.146.190:41961: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":147,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.146.190:41961: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 10:34:46.987644 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 4416
2026-08-08T10:34:47Z chronyd exiting
I20260808 10:34:47.028167 31306 test_util.cc:175] -----------------------------------------------
I20260808 10:34:47.028270 31306 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185256684052-31306-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (364 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 10:34:47.029759 31306 mini_oidc.cc:152] Starting JWKS server
I20260808 10:34:47.030736 31306 webserver.cc:460] Webserver started at https://127.0.0.1:41949/ using document root <none> and password file <none>
I20260808 10:34:47.030861 31306 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:34:47.031183 31306 webserver.cc:460] Webserver started at http://127.0.0.1:41059/ using document root <none> and password file <none>
2026-08-08T10:34:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:47Z Disabled control of system clock
I20260808 10:34:47.042850 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:37363
--webserver_interface=127.30.146.190
--webserver_port=0
--builtin_ntp_servers=127.30.146.148:44751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:37363
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41949/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/testchainca.cert with env {}
W20260808 10:34:47.162719 4529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:47.163043 4529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:47.163122 4529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:47.167308 4529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:47.167404 4529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:47.167434 4529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:47.167474 4529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:47.167531 4529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:34:47.167585 4529 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:34:47.167618 4529 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41949/jwks.json
I20260808 10:34:47.173115 4529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:44751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:37363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:37363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41949/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:47.174342 4529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:47.175434 4529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:47.184549 4539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:47.184873 4529 server_base.cc:1034] running on GCE node
W20260808 10:34:47.184581 4536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:47.184549 4537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:47.185626 4529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:47.186853 4529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:47.188057 4529 hybrid_clock.cc:648] HybridClock initialized: now 1786185287188021 us; error 51 us; skew 500 ppm
I20260808 10:34:47.190256 4529 webserver.cc:460] Webserver started at http://127.30.146.190:36205/ using document root <none> and password file <none>
I20260808 10:34:47.190888 4529 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:47.190953 4529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:47.191208 4529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:47.193400 4529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "6db55cea9bc94db99f200d2f29aa676c"
format_stamp: "Formatted at 2026-08-08 10:34:47 on dist-test-slave-9gcw"
server_key: "0de8a2e5db385ad005c534192bf11399"
server_key_iv: "22bd399919d088a966bde0fadfefeb93"
server_key_version: "encryptionkey@0"
I20260808 10:34:47.194064 4529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "6db55cea9bc94db99f200d2f29aa676c"
format_stamp: "Formatted at 2026-08-08 10:34:47 on dist-test-slave-9gcw"
server_key: "0de8a2e5db385ad005c534192bf11399"
server_key_iv: "22bd399919d088a966bde0fadfefeb93"
server_key_version: "encryptionkey@0"
I20260808 10:34:47.198110 4529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:34:47.200821 4545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:47.202293 4529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:47.202502 4529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "6db55cea9bc94db99f200d2f29aa676c"
format_stamp: "Formatted at 2026-08-08 10:34:47 on dist-test-slave-9gcw"
server_key: "0de8a2e5db385ad005c534192bf11399"
server_key_iv: "22bd399919d088a966bde0fadfefeb93"
server_key_version: "encryptionkey@0"
I20260808 10:34:47.202634 4529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:47.217783 4529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:47.218518 4529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:47.218734 4529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:47.227883 4529 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:37363
I20260808 10:34:47.227923 4604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:37363 every 8 connection(s)
I20260808 10:34:47.229575 4529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:47.232278 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 4529
I20260808 10:34:47.232414 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:47.232719 31306 external_mini_cluster.cc:1428] Setting key 27c288cff11270fa2fef1e3301db39b3
I20260808 10:34:47.234876 4605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:47.240814 4605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c: Bootstrap starting.
I20260808 10:34:47.242660 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:47.234193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.30.146.190:37363)
0808 10:34:47.235000 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:47.235024 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:47.235076 (+ 52us) server_negotiation.cc:407] Connection header received
0808 10:34:47.236071 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:47.236095 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:47.236518 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:47.236860 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:47.237352 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.238357 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.239275 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.239761 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.239932 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:47.240858 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:47.240863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:47.240875 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:47.241382 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:47.241441 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:47.241462 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:47.241569 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:47.241689 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 10:34:47.241954 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":196,"threads_started":1}
I20260808 10:34:47.243357 4605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:47.244128 4605 log.cc:824] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:47.245515 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:34:47.246400 4605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c: No bootstrap required, opened a new log
I20260808 10:34:47.250125 4605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db55cea9bc94db99f200d2f29aa676c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37363 } }
I20260808 10:34:47.250403 4605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:47.250495 4605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6db55cea9bc94db99f200d2f29aa676c, State: Initialized, Role: FOLLOWER
I20260808 10:34:47.250974 4605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db55cea9bc94db99f200d2f29aa676c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37363 } }
I20260808 10:34:47.251124 4605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:47.251200 4605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:47.251305 4605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:47.252296 4605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db55cea9bc94db99f200d2f29aa676c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37363 } }
I20260808 10:34:47.252641 4605 leader_election.cc:302] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6db55cea9bc94db99f200d2f29aa676c; no voters:
I20260808 10:34:47.252964 4605 leader_election.cc:288] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:47.253121 4611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:47.253396 4611 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [term 1 LEADER]: Becoming Leader. State: Replica: 6db55cea9bc94db99f200d2f29aa676c, State: Running, Role: LEADER
I20260808 10:34:47.253734 4611 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db55cea9bc94db99f200d2f29aa676c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37363 } }
I20260808 10:34:47.254319 4605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:47.255565 4612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6db55cea9bc94db99f200d2f29aa676c. Latest consensus state: current_term: 1 leader_uuid: "6db55cea9bc94db99f200d2f29aa676c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db55cea9bc94db99f200d2f29aa676c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37363 } } }
I20260808 10:34:47.255573 4613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6db55cea9bc94db99f200d2f29aa676c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6db55cea9bc94db99f200d2f29aa676c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37363 } } }
I20260808 10:34:47.255716 4613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:47.255718 4612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:47.256794 4620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:47.259649 4620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:47.265522 4620 catalog_manager.cc:1347] Generated new cluster ID: ed8035d3482a4910957d99fa612a69a3
I20260808 10:34:47.265610 4620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:47.278043 4620 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:47.279047 4620 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:47.283586 4620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6db55cea9bc94db99f200d2f29aa676c: Generated new TSK 0
I20260808 10:34:47.284379 4620 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 10:34:47.315137 4638 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 10:34:47.315241 4638 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 10:34:47.315989 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:47.299469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38602 (local address 127.30.146.190:37363)
0808 10:34:47.299624 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:47.299628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:47.299823 (+ 195us) server_negotiation.cc:407] Connection header received
0808 10:34:47.299996 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:47.299998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:47.300048 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:47.300142 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:34:47.300478 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.301028 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.302439 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.302652 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.302738 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:47.303149 (+ 411us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:34:47.315614 (+ 12465us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41949/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 10:34:47.315857 (+ 243us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38602: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41949/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 10:34:47.316082 4609 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38602: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41949/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 10:34:47.316370 4637 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:34:47.299304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:37363 (local address 127.0.0.1:38602)
0808 10:34:47.299691 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:47.299711 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:47.299855 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:47.300185 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:47.300189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:47.300199 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:34:47.300346 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:47.300355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.301179 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.301184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:47.302291 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:47.302305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.302796 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.302802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:47.302926 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:47.302963 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:34:47.315867 (+ 12904us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41949/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 10:34:47.316121 (+ 254us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.146.190:37363: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41949/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":115,"threads_started":1}
I20260808 10:34:47.317164 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 4529
2026-08-08T10:34:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (332 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 10:34:47.362066 31306 mini_oidc.cc:152] Starting JWKS server
I20260808 10:34:47.363157 31306 webserver.cc:460] Webserver started at https://127.0.0.1:33099/ using document root <none> and password file <none>
I20260808 10:34:47.363318 31306 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:34:47.363646 31306 webserver.cc:460] Webserver started at http://127.0.0.1:33031/ using document root <none> and password file <none>
2026-08-08T10:34:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:47Z Disabled control of system clock
I20260808 10:34:47.375304 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:46591
--webserver_interface=127.30.146.190
--webserver_port=0
--builtin_ntp_servers=127.30.146.148:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:46591
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33099/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 10:34:47.489854 4644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:47.490147 4644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:47.490190 4644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:47.494295 4644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:47.494361 4644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:47.494377 4644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:47.494395 4644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:47.494410 4644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:34:47.494427 4644 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:34:47.494480 4644 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33099/jwks.json
I20260808 10:34:47.499416 4644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:46591
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:46591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33099/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:47.500620 4644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:47.502122 4644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:47.510974 4651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:47.511008 4653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:47.511186 4650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:47.511253 4644 server_base.cc:1034] running on GCE node
I20260808 10:34:47.511911 4644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:47.513183 4644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:47.514385 4644 hybrid_clock.cc:648] HybridClock initialized: now 1786185287514372 us; error 26 us; skew 500 ppm
I20260808 10:34:47.516331 4644 webserver.cc:460] Webserver started at http://127.30.146.190:45453/ using document root <none> and password file <none>
I20260808 10:34:47.516940 4644 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:47.517035 4644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:47.517308 4644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:47.520099 4644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "5dc706da888e4861937032d64ada4de4"
format_stamp: "Formatted at 2026-08-08 10:34:47 on dist-test-slave-9gcw"
server_key: "8f6175b3997092c9ba11f8c9627f4b35"
server_key_iv: "f97e674ea04221c084834980a39387be"
server_key_version: "encryptionkey@0"
I20260808 10:34:47.520807 4644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "5dc706da888e4861937032d64ada4de4"
format_stamp: "Formatted at 2026-08-08 10:34:47 on dist-test-slave-9gcw"
server_key: "8f6175b3997092c9ba11f8c9627f4b35"
server_key_iv: "f97e674ea04221c084834980a39387be"
server_key_version: "encryptionkey@0"
I20260808 10:34:47.525890 4644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 10:34:47.528539 4660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:47.529922 4644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:34:47.530066 4644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "5dc706da888e4861937032d64ada4de4"
format_stamp: "Formatted at 2026-08-08 10:34:47 on dist-test-slave-9gcw"
server_key: "8f6175b3997092c9ba11f8c9627f4b35"
server_key_iv: "f97e674ea04221c084834980a39387be"
server_key_version: "encryptionkey@0"
I20260808 10:34:47.530196 4644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:47.543249 4644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:47.543974 4644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:47.544185 4644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:47.551970 4644 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:46591
I20260808 10:34:47.551992 4718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:46591 every 8 connection(s)
I20260808 10:34:47.552999 4644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:47.556414 4719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:47.562063 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 4644
I20260808 10:34:47.562208 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:47.562263 4719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4: Bootstrap starting.
I20260808 10:34:47.562530 31306 external_mini_cluster.cc:1428] Setting key a54b5f99b35ab8e3903bd2e34855611f
I20260808 10:34:47.565026 4719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:47.565832 4719 log.cc:824] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:47.567736 4719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4: No bootstrap required, opened a new log
I20260808 10:34:47.571650 4719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc706da888e4861937032d64ada4de4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46591 } }
I20260808 10:34:47.571936 4719 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:47.572021 4719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc706da888e4861937032d64ada4de4, State: Initialized, Role: FOLLOWER
I20260808 10:34:47.572441 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:47.563961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43832 (local address 127.30.146.190:46591)
0808 10:34:47.564696 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:47.564711 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:47.564748 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:34:47.565488 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:47.565508 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:47.565955 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:47.566301 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:47.566734 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.567702 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.568658 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.569241 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.569422 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:47.570352 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:47.570358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:47.570371 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:47.570768 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:47.570841 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:47.570861 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:47.570976 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:47.571144 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 10:34:47.571418 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":142,"threads_started":1}
I20260808 10:34:47.572467 4719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc706da888e4861937032d64ada4de4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46591 } }
I20260808 10:34:47.572679 4719 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:47.572770 4719 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:47.572873 4719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:47.573876 4719 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc706da888e4861937032d64ada4de4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46591 } }
I20260808 10:34:47.574227 4719 leader_election.cc:302] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dc706da888e4861937032d64ada4de4; no voters:
I20260808 10:34:47.574579 4719 leader_election.cc:288] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:47.574688 4724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:47.574954 4724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [term 1 LEADER]: Becoming Leader. State: Replica: 5dc706da888e4861937032d64ada4de4, State: Running, Role: LEADER
I20260808 10:34:47.575268 4724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc706da888e4861937032d64ada4de4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46591 } }
I20260808 10:34:47.575407 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:34:47.575845 4719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:47.576521 4724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dc706da888e4861937032d64ada4de4. Latest consensus state: current_term: 1 leader_uuid: "5dc706da888e4861937032d64ada4de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc706da888e4861937032d64ada4de4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46591 } } }
I20260808 10:34:47.576640 4724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:47.577246 4726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dc706da888e4861937032d64ada4de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc706da888e4861937032d64ada4de4" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46591 } } }
I20260808 10:34:47.577406 4726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4 [sys.catalog]: This master's current role is: LEADER
W20260808 10:34:47.580962 4740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:34:47.581053 4740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:34:47.581126 4733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:47.582015 4733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:47.587870 4733 catalog_manager.cc:1347] Generated new cluster ID: 1d355f4de39d40a0a9c765bf3d1ae3ad
I20260808 10:34:47.587944 4733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:47.613937 4733 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:47.615391 4733 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:47.624199 4733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dc706da888e4861937032d64ada4de4: Generated new TSK 0
I20260808 10:34:47.625036 4733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:34:47.646361 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:47.641052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.30.146.190:46591)
0808 10:34:47.641281 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:47.641286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:47.641474 (+ 188us) server_negotiation.cc:407] Connection header received
0808 10:34:47.641609 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:47.641611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:47.641662 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:47.641758 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:34:47.642040 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.642600 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.643926 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.644138 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.644214 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:47.644562 (+ 348us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:34:47.645904 (+ 1342us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33099/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 10:34:47.646174 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43836: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33099/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":113}
W20260808 10:34:47.646389 4749 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:34:47.640973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46591 (local address 127.0.0.1:43836)
0808 10:34:47.641342 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:47.641358 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:47.641484 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:47.641784 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:47.641787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:47.641795 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:34:47.641927 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:47.641934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.642727 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.642731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:47.643805 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:47.643816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.644259 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.644263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:47.644354 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:47.644385 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:34:47.646077 (+ 1692us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33099/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 10:34:47.646228 (+ 151us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.146.190:46591: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33099/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":176,"threads_started":1}
W20260808 10:34:47.646822 4722 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43836: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33099/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260808 10:34:47.647087 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 4644
E20260808 10:34:47.658879 4751 webserver.cc:533] Webserver: sslize failed:
E20260808 10:34:47.658964 4751 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T10:34:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (328 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T10:34:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:47Z Disabled control of system clock
I20260808 10:34:47.701071 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:35497
--webserver_interface=127.30.146.190
--webserver_port=0
--builtin_ntp_servers=127.30.146.148:39023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:35497
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/testchainca.cert with env {}
W20260808 10:34:47.816882 4755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:47.817224 4755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:47.817330 4755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:47.821672 4755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:47.821748 4755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:47.821762 4755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:47.821787 4755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:47.821803 4755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:34:47.821820 4755 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:34:47.821867 4755 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 10:34:47.827586 4755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:39023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:35497
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:35497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:47.828763 4755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:47.829892 4755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:47.839797 4760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:47.839874 4761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:47.839813 4763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:47.840406 4755 server_base.cc:1034] running on GCE node
I20260808 10:34:47.841075 4755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:47.842298 4755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:47.843503 4755 hybrid_clock.cc:648] HybridClock initialized: now 1786185287843490 us; error 37 us; skew 500 ppm
I20260808 10:34:47.845734 4755 webserver.cc:460] Webserver started at http://127.30.146.190:35055/ using document root <none> and password file <none>
I20260808 10:34:47.846354 4755 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:47.846455 4755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:47.846709 4755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:47.848922 4755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "2742174995d648ac9a8a75666729813c"
format_stamp: "Formatted at 2026-08-08 10:34:47 on dist-test-slave-9gcw"
server_key: "6034e4134751740f90ec9d30fab0845c"
server_key_iv: "062c4d5f24ee6fa96efd9bc16c75192d"
server_key_version: "encryptionkey@0"
I20260808 10:34:47.849591 4755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "2742174995d648ac9a8a75666729813c"
format_stamp: "Formatted at 2026-08-08 10:34:47 on dist-test-slave-9gcw"
server_key: "6034e4134751740f90ec9d30fab0845c"
server_key_iv: "062c4d5f24ee6fa96efd9bc16c75192d"
server_key_version: "encryptionkey@0"
I20260808 10:34:47.853863 4755 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:34:47.856621 4769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:47.858104 4755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 10:34:47.858274 4755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "2742174995d648ac9a8a75666729813c"
format_stamp: "Formatted at 2026-08-08 10:34:47 on dist-test-slave-9gcw"
server_key: "6034e4134751740f90ec9d30fab0845c"
server_key_iv: "062c4d5f24ee6fa96efd9bc16c75192d"
server_key_version: "encryptionkey@0"
I20260808 10:34:47.858415 4755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:47.882226 4755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:47.883034 4755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:47.883262 4755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:47.890987 4755 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:35497
I20260808 10:34:47.891008 4829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:35497 every 8 connection(s)
I20260808 10:34:47.892015 4755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:47.895040 4830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:47.897925 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 4755
I20260808 10:34:47.898088 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:47.898403 31306 external_mini_cluster.cc:1428] Setting key 4a1ece396d7b5e25bac6b71ad09aae76
I20260808 10:34:47.901641 4830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c: Bootstrap starting.
I20260808 10:34:47.904083 4830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:47.904893 4830 log.cc:824] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:47.907397 4830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c: No bootstrap required, opened a new log
I20260808 10:34:47.908830 4833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:47.899952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36352 (local address 127.30.146.190:35497)
0808 10:34:47.900787 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:47.900825 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:47.900864 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:34:47.901557 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:47.901576 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:47.902044 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:47.902398 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:47.902870 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.903866 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.904856 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.905568 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.905769 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:47.906736 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:47.906741 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:47.906754 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:34:47.907155 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:47.907213 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:34:47.907233 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:47.907327 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:47.907815 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 10:34:47.908059 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":252,"threads_started":1}
I20260808 10:34:47.910746 4830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2742174995d648ac9a8a75666729813c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35497 } }
I20260808 10:34:47.910985 4830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:47.911047 4830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2742174995d648ac9a8a75666729813c, State: Initialized, Role: FOLLOWER
I20260808 10:34:47.911561 4830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2742174995d648ac9a8a75666729813c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35497 } }
I20260808 10:34:47.911732 4830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:47.911826 4830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:47.911945 4830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:47.912606 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:34:47.913123 4830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2742174995d648ac9a8a75666729813c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35497 } }
I20260808 10:34:47.913530 4830 leader_election.cc:302] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2742174995d648ac9a8a75666729813c; no voters:
I20260808 10:34:47.913901 4830 leader_election.cc:288] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:47.914047 4835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:47.914312 4835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [term 1 LEADER]: Becoming Leader. State: Replica: 2742174995d648ac9a8a75666729813c, State: Running, Role: LEADER
I20260808 10:34:47.914870 4835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2742174995d648ac9a8a75666729813c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35497 } }
I20260808 10:34:47.915189 4830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:47.916179 4836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2742174995d648ac9a8a75666729813c. Latest consensus state: current_term: 1 leader_uuid: "2742174995d648ac9a8a75666729813c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2742174995d648ac9a8a75666729813c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35497 } } }
I20260808 10:34:47.916344 4836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:47.916694 4837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2742174995d648ac9a8a75666729813c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2742174995d648ac9a8a75666729813c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35497 } } }
I20260808 10:34:47.916812 4837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:47.916770 4844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:47.920150 4844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:47.926918 4844 catalog_manager.cc:1347] Generated new cluster ID: 7e34f60ff9794ddb86a77623869a9a47
I20260808 10:34:47.927016 4844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:47.944540 4844 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:47.945884 4844 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:47.958308 4844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2742174995d648ac9a8a75666729813c: Generated new TSK 0
I20260808 10:34:47.959054 4844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:34:48.395905 4833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:47.973952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.30.146.190:35497)
0808 10:34:47.974141 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:47.974146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:47.974317 (+ 171us) server_negotiation.cc:407] Connection header received
0808 10:34:47.974491 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:47.974494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:47.974555 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:47.974674 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:34:47.974938 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.975800 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.976808 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.977198 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.977305 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:47.977629 (+ 324us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:34:48.395496 (+417867us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 10:34:48.395776 (+ 280us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36366: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":75}
W20260808 10:34:48.395982 4833 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36366: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 10:34:48.396210 4861 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:34:47.973833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35497 (local address 127.0.0.1:36366)
0808 10:34:47.974198 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:47.974213 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:47.974342 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:47.974707 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:47.974709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:47.974717 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:34:47.974818 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:47.974824 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.975933 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.975936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:47.976676 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:47.976688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:47.977330 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:47.977338 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:47.977414 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:47.977440 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:34:48.395750 (+418310us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 10:34:48.395944 (+ 194us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.146.190:35497: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":126,"threads_started":1}
I20260808 10:34:48.396926 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 4755
2026-08-08T10:34:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (729 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (132 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (131 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:48 dist-test-slave-9gcw krb5kdc[4874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:48 dist-test-slave-9gcw krb5kdc[4874](info): set up 2 sockets
Aug 08 10:34:48 dist-test-slave-9gcw krb5kdc[4874](info): commencing operation
krb5kdc: starting...
W20260808 10:34:49.815446 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:49 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:34:49.958038 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:42475
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:42475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:50.079756 4888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:50.080119 4888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:50.080201 4888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:50.084455 4888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:50.084550 4888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:50.084589 4888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:50.084666 4888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:50.084707 4888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:50.090462 4888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:42475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:42475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:50.091658 4888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:50.092751 4888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:50.101967 4894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:50.101989 4897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:50.102044 4895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:50.102928 4888 server_base.cc:1034] running on GCE node
I20260808 10:34:50.103536 4888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:50.104830 4888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:50.106027 4888 hybrid_clock.cc:648] HybridClock initialized: now 1786185290105991 us; error 59 us; skew 500 ppm
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:50.117456 4888 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:34:50.118521 4888 webserver.cc:460] Webserver started at http://127.30.146.190:39019/ using document root <none> and password file <none>
I20260808 10:34:50.119156 4888 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:50.119259 4888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:50.119519 4888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:50.121886 4888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "ff38f551ede44b32ae66f63228ce66c8"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "ba187285e11b7a9089e44cb827dc82a3"
server_key_iv: "115e51c695db8b046fdb79c624c5b201"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.122486 4888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "ff38f551ede44b32ae66f63228ce66c8"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "ba187285e11b7a9089e44cb827dc82a3"
server_key_iv: "115e51c695db8b046fdb79c624c5b201"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.126466 4888 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:50.128993 4905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:50.130323 4888 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:34:50.130471 4888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "ff38f551ede44b32ae66f63228ce66c8"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "ba187285e11b7a9089e44cb827dc82a3"
server_key_iv: "115e51c695db8b046fdb79c624c5b201"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.130590 4888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:50.148226 4888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:50.151749 4888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:50.151990 4888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:50.159787 4888 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:42475
I20260808 10:34:50.159821 4971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:42475 every 8 connection(s)
I20260808 10:34:50.160841 4888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:50.163915 4972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:50.165969 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 4888
I20260808 10:34:50.166086 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:50.166424 31306 external_mini_cluster.cc:1428] Setting key 903258afcb3150baa3ce66920df6a889
I20260808 10:34:50.171486 4972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8: Bootstrap starting.
I20260808 10:34:50.174386 4972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:50.175348 4972 log.cc:824] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:50.177767 4972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8: No bootstrap required, opened a new log
I20260808 10:34:50.181600 4972 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff38f551ede44b32ae66f63228ce66c8" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42475 } }
I20260808 10:34:50.181839 4972 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:50.181910 4972 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff38f551ede44b32ae66f63228ce66c8, State: Initialized, Role: FOLLOWER
I20260808 10:34:50.182371 4972 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff38f551ede44b32ae66f63228ce66c8" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42475 } }
I20260808 10:34:50.182427 4975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:50.167938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50366 (local address 127.30.146.190:42475)
0808 10:34:50.168376 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:50.168399 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:50.168447 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:34:50.169426 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:50.169453 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:50.169886 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:50.170282 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:50.171015 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.172472 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.173405 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.174143 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.174350 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:50.177251 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:50.177258 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:50.177272 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:50.177302 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:50.179285 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:50.179762 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:50.179769 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:50.179781 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:50.179894 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:50.180197 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:50.180201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:50.180203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:50.180635 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:50.180859 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:50.181136 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 10:34:50.181463 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":109,"threads_started":1}
I20260808 10:34:50.182494 4972 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:50.182572 4972 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:50.182709 4972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:50.183785 4972 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff38f551ede44b32ae66f63228ce66c8" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42475 } }
I20260808 10:34:50.184156 4972 leader_election.cc:302] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff38f551ede44b32ae66f63228ce66c8; no voters:
I20260808 10:34:50.184484 4972 leader_election.cc:288] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:50.184731 4977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:50.184919 4977 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [term 1 LEADER]: Becoming Leader. State: Replica: ff38f551ede44b32ae66f63228ce66c8, State: Running, Role: LEADER
I20260808 10:34:50.185326 4977 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff38f551ede44b32ae66f63228ce66c8" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42475 } }
I20260808 10:34:50.185886 4972 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:50.187065 4978 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff38f551ede44b32ae66f63228ce66c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff38f551ede44b32ae66f63228ce66c8" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42475 } } }
I20260808 10:34:50.187213 4978 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:50.187533 4979 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff38f551ede44b32ae66f63228ce66c8. Latest consensus state: current_term: 1 leader_uuid: "ff38f551ede44b32ae66f63228ce66c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff38f551ede44b32ae66f63228ce66c8" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42475 } } }
I20260808 10:34:50.187633 4979 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8 [sys.catalog]: This master's current role is: LEADER
W20260808 10:34:50.191102 4992 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:34:50.191206 4992 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:34:50.191273 4986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:50.192247 4986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:50.197988 4986 catalog_manager.cc:1347] Generated new cluster ID: 542293d5f08e4c53962f803c53a28014
I20260808 10:34:50.198066 4986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:50.209846 4986 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:50.211205 4986 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:50.221024 4986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff38f551ede44b32ae66f63228ce66c8: Generated new TSK 0
I20260808 10:34:50.221822 4986 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:34:50.252674 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42475
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:50.376071 4998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:50.376361 4998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:50.376410 4998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:50.380752 4998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:50.380817 4998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:50.380877 4998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:50.386281 4998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:50.387411 4998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:50.388461 4998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:50.398775 5007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:50.398814 5004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:50.398788 5003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:50.399034 4998 server_base.cc:1034] running on GCE node
I20260808 10:34:50.399468 4998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:50.400146 4998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:50.401377 4998 hybrid_clock.cc:648] HybridClock initialized: now 1786185290401360 us; error 33 us; skew 500 ppm
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:50.404594 4998 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:34:50.405720 4998 webserver.cc:460] Webserver started at http://127.30.146.129:35261/ using document root <none> and password file <none>
I20260808 10:34:50.406358 4998 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:50.406430 4998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:50.406687 4998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:50.409013 4998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "d03b2638594b4506afa96f3bfa9dd558"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "191b6f49e76386af0cef44e35c95dbcf"
server_key_iv: "03c468b132bc7a95104beaf57e6451f7"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.409783 4998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "d03b2638594b4506afa96f3bfa9dd558"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "191b6f49e76386af0cef44e35c95dbcf"
server_key_iv: "03c468b132bc7a95104beaf57e6451f7"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.414134 4998 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:50.416960 5014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:50.418432 4998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:50.418588 4998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "d03b2638594b4506afa96f3bfa9dd558"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "191b6f49e76386af0cef44e35c95dbcf"
server_key_iv: "03c468b132bc7a95104beaf57e6451f7"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.418717 4998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:50.432641 4998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:50.436453 4998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:50.436681 4998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:50.437373 4998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:50.438268 4998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:50.438345 4998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:50.438438 4998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:50.438490 4998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:50.448127 4998 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:44797
I20260808 10:34:50.448153 5137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:44797 every 8 connection(s)
I20260808 10:34:50.449083 4998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:50.450501 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 4998
I20260808 10:34:50.450632 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:50.451005 31306 external_mini_cluster.cc:1428] Setting key 33314563cd49ac8526c56ec976bff1e5
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:34:50.465771 4975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:50.451250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:60721 (local address 127.30.146.190:42475)
0808 10:34:50.451404 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:50.451407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:50.452404 (+ 997us) server_negotiation.cc:407] Connection header received
0808 10:34:50.454045 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:50.454052 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:50.454121 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:50.454229 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:50.455662 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.456229 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.458135 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.458481 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.458574 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:50.461774 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:50.461778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:50.461780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:50.461806 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:50.463342 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:50.463957 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:50.463960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:50.463962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:50.464039 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:50.464406 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:50.464409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:50.464411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:50.464611 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:50.464720 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:50.465441 (+ 721us) server_negotiation.cc:299] Negotiation successful
0808 10:34:50.465582 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:34:50.466612 5140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:50.451731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:42475 (local address 127.30.146.129:60721)
0808 10:34:50.452227 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:50.452285 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:50.453806 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:50.454358 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:50.454365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:50.454737 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:50.455484 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:50.455506 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.456373 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.456379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:50.457974 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:50.457989 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.458667 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.458673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:50.458919 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:50.459644 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:50.459672 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:50.461515 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:50.463474 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:50.463480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:50.463497 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:50.463837 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:50.464141 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:50.464145 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:50.464147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:50.464302 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:50.464722 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:50.464731 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:50.465083 (+ 352us) client_negotiation.cc:769] Sending connection context
0808 10:34:50.465339 (+ 256us) client_negotiation.cc:240] Negotiation successful
0808 10:34:50.465724 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":102,"threads_started":1}
I20260808 10:34:50.468081 5138 heartbeater.cc:347] Connected to a master server at 127.30.146.190:42475
I20260808 10:34:50.468420 5138 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:50.469113 5138 heartbeater.cc:511] Master 127.30.146.190:42475 requested a full tablet report, sending...
I20260808 10:34:50.471226 4923 ts_manager.cc:194] Registered new tserver with Master: d03b2638594b4506afa96f3bfa9dd558 (127.30.146.129:44797)
I20260808 10:34:50.472524 4923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:60721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:34:50.488265 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42475
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:50.608851 5143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:50.609205 5143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:50.609299 5143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:50.613629 5143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:50.613933 5143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:50.614148 5143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:50.620959 5143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:50.622344 5143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:50.623565 5143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:34:50.633978 5143 server_base.cc:1034] running on GCE node
W20260808 10:34:50.633961 5150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:50.633939 5152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:50.633939 5148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:50.634604 5143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:50.635506 5143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:50.636718 5143 hybrid_clock.cc:648] HybridClock initialized: now 1786185290636681 us; error 47 us; skew 500 ppm
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:50.639932 5143 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:34:50.640985 5143 webserver.cc:460] Webserver started at http://127.30.146.130:38547/ using document root <none> and password file <none>
I20260808 10:34:50.641736 5143 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:50.641810 5143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:50.642001 5143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:50.644215 5143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "7db8ec86a13040c5937128d19dbba115"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "f53d0d4dd82ffc7eec6a5ff2e356f9a8"
server_key_iv: "0cf76665753eca38c8cea1dd48d6089d"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.644788 5143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "7db8ec86a13040c5937128d19dbba115"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "f53d0d4dd82ffc7eec6a5ff2e356f9a8"
server_key_iv: "0cf76665753eca38c8cea1dd48d6089d"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.649255 5143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:50.652034 5159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:50.653541 5143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:50.653702 5143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "7db8ec86a13040c5937128d19dbba115"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "f53d0d4dd82ffc7eec6a5ff2e356f9a8"
server_key_iv: "0cf76665753eca38c8cea1dd48d6089d"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.653820 5143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:50.684285 5143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:50.688409 5143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:50.688663 5143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:50.689424 5143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:50.690369 5143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:50.690460 5143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:50.690538 5143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:50.690582 5143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:50.700714 5143 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:38859
I20260808 10:34:50.700800 5285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:38859 every 8 connection(s)
I20260808 10:34:50.701815 5143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:50.707188 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 5143
I20260808 10:34:50.707337 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:50.707626 31306 external_mini_cluster.cc:1428] Setting key df172767f205d654c64075d8c97cd382
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:50.717566 4975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:50.703902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:36041 (local address 127.30.146.190:42475)
0808 10:34:50.704074 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:50.704078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:50.704913 (+ 835us) server_negotiation.cc:407] Connection header received
0808 10:34:50.706406 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:50.706409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:50.706472 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:50.706570 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:50.707948 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.708590 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.709982 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.710214 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.710330 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:50.713364 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:50.713367 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:50.713369 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:50.713394 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:50.715058 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:50.715782 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:50.715785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:50.715786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:50.715869 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:50.716313 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:50.716316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:50.716317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:50.716494 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:50.716571 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:50.717331 (+ 760us) server_negotiation.cc:299] Negotiation successful
0808 10:34:50.717439 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 10:34:50.718874 5290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:50.704182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:42475 (local address 127.30.146.130:36041)
0808 10:34:50.704749 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:50.704796 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:50.706170 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:50.706755 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:50.706762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:50.707168 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:50.707792 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:50.707812 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.708723 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.708727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:50.709852 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:50.709863 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.710374 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.710378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:50.710561 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:50.711284 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:50.711301 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:50.713108 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:50.715212 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:50.715222 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:50.715242 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:50.715651 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:50.715971 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:50.715976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:50.715981 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:50.716204 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:50.716607 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:50.716617 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:50.716992 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 10:34:50.717277 (+ 285us) client_negotiation.cc:240] Negotiation successful
0808 10:34:50.717674 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:34:50.720275 5287 heartbeater.cc:347] Connected to a master server at 127.30.146.190:42475
I20260808 10:34:50.720542 5287 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:50.721191 5287 heartbeater.cc:511] Master 127.30.146.190:42475 requested a full tablet report, sending...
I20260808 10:34:50.722401 4923 ts_manager.cc:194] Registered new tserver with Master: 7db8ec86a13040c5937128d19dbba115 (127.30.146.130:38859)
I20260808 10:34:50.723306 4923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:36041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:34:50.742779 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42475
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:50.874841 5293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:50.875131 5293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:50.875180 5293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:50.879391 5293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:50.879472 5293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:50.879534 5293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:50.884984 5293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:50.886188 5293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:50.887337 5293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:50.897538 5301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:50.897612 5299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:50.897701 5298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:50.897831 5293 server_base.cc:1034] running on GCE node
I20260808 10:34:50.898244 5293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:50.898962 5293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:50.900202 5293 hybrid_clock.cc:648] HybridClock initialized: now 1786185290900176 us; error 44 us; skew 500 ppm
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:50.903126 5293 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:34:50.904045 5293 webserver.cc:460] Webserver started at http://127.30.146.131:45369/ using document root <none> and password file <none>
I20260808 10:34:50.904621 5293 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:50.904696 5293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:50.904933 5293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:50.907120 5293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "5a28bcca27c249e9a10d7bcb0056b2c3"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "7dbb5afc7eec0da4243207e5e4b2fcee"
server_key_iv: "cc54d6ac2b6d63ca6c62b0a47f5a47f0"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.907783 5293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "5a28bcca27c249e9a10d7bcb0056b2c3"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "7dbb5afc7eec0da4243207e5e4b2fcee"
server_key_iv: "cc54d6ac2b6d63ca6c62b0a47f5a47f0"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.911755 5293 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:34:50.914477 5309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:50.915695 5293 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:34:50.915832 5293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "5a28bcca27c249e9a10d7bcb0056b2c3"
format_stamp: "Formatted at 2026-08-08 10:34:50 on dist-test-slave-9gcw"
server_key: "7dbb5afc7eec0da4243207e5e4b2fcee"
server_key_iv: "cc54d6ac2b6d63ca6c62b0a47f5a47f0"
server_key_version: "encryptionkey@0"
I20260808 10:34:50.915949 5293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:50.928800 5293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:50.932777 5293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:50.932994 5293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:50.933686 5293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:50.934639 5293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:50.934705 5293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:50.934772 5293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:50.934803 5293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:50.944164 5293 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:46709
I20260808 10:34:50.944199 5440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:46709 every 8 connection(s)
I20260808 10:34:50.945081 5293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:50.951589 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 5293
I20260808 10:34:50.951721 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:50.952051 31306 external_mini_cluster.cc:1428] Setting key 579170d654c6278e0e182dcfce98d6c4
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:34:50.960429 4975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:50.947080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:35935 (local address 127.30.146.190:42475)
0808 10:34:50.947227 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:50.947231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:50.948083 (+ 852us) server_negotiation.cc:407] Connection header received
0808 10:34:50.949608 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:50.949611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:50.949683 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:50.949791 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:50.950998 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.951718 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.953265 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.953570 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.953685 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:50.956626 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:50.956629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:50.956631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:50.956656 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:50.958253 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:50.958926 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:50.958929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:50.958930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:50.959006 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:50.959410 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:50.959413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:50.959414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:50.959592 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:50.959668 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:50.960197 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 10:34:50.960304 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:34:50.961416 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:50.947467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:42475 (local address 127.30.146.131:35935)
0808 10:34:50.947929 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:50.947971 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:50.949392 (+ 1421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:50.949964 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:50.949972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:50.950367 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:50.950848 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:50.950862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.951900 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.951903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:50.953068 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:50.953081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.953794 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.953799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:50.954015 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:50.954779 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:50.954798 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:50.956446 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:50.958391 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:50.958397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:50.958411 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:50.958806 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:50.959143 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:50.959147 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:50.959153 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:50.959318 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:50.959700 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:50.959705 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:50.959957 (+ 252us) client_negotiation.cc:769] Sending connection context
0808 10:34:50.960157 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 10:34:50.960443 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
I20260808 10:34:50.962605 5442 heartbeater.cc:347] Connected to a master server at 127.30.146.190:42475
I20260808 10:34:50.962863 5442 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:50.963394 5442 heartbeater.cc:511] Master 127.30.146.190:42475 requested a full tablet report, sending...
I20260808 10:34:50.964447 4923 ts_manager.cc:194] Registered new tserver with Master: 5a28bcca27c249e9a10d7bcb0056b2c3 (127.30.146.131:46709)
I20260808 10:34:50.964998 4923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:35935
I20260808 10:34:50.966511 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/test-admin.keytab.
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:50.986912 31306 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 10:34:50.990057 5449 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:34:50.988558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36204 (local address 127.30.146.129:44797)
0808 10:34:50.988881 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:50.988887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:50.988911 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:34:50.988963 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:50.988969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:50.989224 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:50.989344 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:50.989874 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36204: BlockingRecv error: recv got EOF from 127.0.0.1:36204 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":113,"threads_started":1}
W20260808 10:34:50.990187 31306 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:50.991453 31306 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:50.992421 31306 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:34:50 dist-test-slave-9gcw krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:34:51.002835 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:50.993395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.30.146.129:44797)
0808 10:34:50.993525 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:50.993529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:50.993580 (+ 51us) server_negotiation.cc:407] Connection header received
0808 10:34:50.993692 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:50.993694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:50.993736 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:50.993806 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:50.994302 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.995090 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.995949 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:50.996362 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:50.996446 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:50.998806 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:50.998816 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:50.998829 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:50.998851 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:51.000558 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:51.001099 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:51.001105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:51.001107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:51.001298 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:51.001563 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:51.001566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:51.001567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:51.001914 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:51.002112 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:51.002371 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 10:34:51.002605 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 10:34:51.007321 5449 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:34:51.006154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36226 (local address 127.30.146.129:44797)
0808 10:34:51.006286 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:51.006289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:51.006303 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:34:51.006426 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:51.006428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:51.006470 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:51.006695 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:51.007195 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36226: BlockingRecv error: recv got EOF from 127.0.0.1:36226 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 10:34:51.007478 31306 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:34:51 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:51.008724 31306 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:34:51 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:51.009707 31306 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:34:51 dist-test-slave-9gcw krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:34:51.019066 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:51.010562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36238 (local address 127.30.146.129:44797)
0808 10:34:51.010717 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:51.010722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:51.010736 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:34:51.010849 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:51.010852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:51.010887 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:51.010954 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:51.011438 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:51.011949 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:51.012760 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:51.012991 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:51.013082 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:51.015711 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:51.015715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:51.015717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:51.015744 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:51.017415 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:51.017879 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:51.017882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:51.017883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:51.017955 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:51.018189 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:51.018192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:51.018193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:51.018348 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:51.018464 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:51.018801 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 10:34:51.018899 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 10:34:51.022087 5449 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:34:51.021207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36252 (local address 127.30.146.129:44797)
0808 10:34:51.021320 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:51.021324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:51.021344 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:34:51.021441 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:51.021443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:51.021478 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:51.021540 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:51.021983 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36252: BlockingRecv error: recv got EOF from 127.0.0.1:36252 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 10:34:51.022262 31306 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185256684052-31306-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:34:51 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:51.023427 31306 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:34:51 dist-test-slave-9gcw krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:51.024396 31306 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:34:51 dist-test-slave-9gcw krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:34:51.033529 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:51.025334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.30.146.129:44797)
0808 10:34:51.025470 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:51.025475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:51.025669 (+ 194us) server_negotiation.cc:407] Connection header received
0808 10:34:51.025717 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:51.025720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:51.025757 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:51.025845 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:51.026342 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:51.026845 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:51.027630 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:51.027871 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:51.027974 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:51.030482 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:51.030485 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:51.030488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:51.030510 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:51.031778 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:51.032222 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:51.032225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:51.032227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:51.032318 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:51.032602 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:51.032606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:51.032608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:51.032789 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:51.032983 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:51.033222 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 10:34:51.033346 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:34:51.034659 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 4998
I20260808 10:34:51.043171 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 5143
I20260808 10:34:51.050714 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 5293
I20260808 10:34:51.060153 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 4888
2026-08-08T10:34:51Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2400 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:51 dist-test-slave-9gcw krb5kdc[5464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:51 dist-test-slave-9gcw krb5kdc[5464](info): set up 2 sockets
Aug 08 10:34:51 dist-test-slave-9gcw krb5kdc[5464](info): commencing operation
krb5kdc: starting...
W20260808 10:34:52.177356 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:52 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.190.keytab.
Entry for principal oryx/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.190.keytab.
I20260808 10:34:52.325135 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:34753
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.190.keytab
--principal=oryx/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:34753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:52.441129 5485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:52.441425 5485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:52.441468 5485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:52.445536 5485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:52.445600 5485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:52.445618 5485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:52.445636 5485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:52.445650 5485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:52.451027 5485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:34753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.190.keytab
--principal=oryx/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:34753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:52.452179 5485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:52.453318 5485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:34:52.462389 5485 server_base.cc:1034] running on GCE node
W20260808 10:34:52.462352 5493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:52.462312 5491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:52.462314 5490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:52.463198 5485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:52.464385 5485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:52.465579 5485 hybrid_clock.cc:648] HybridClock initialized: now 1786185292465559 us; error 32 us; skew 500 ppm
Aug 08 10:34:52 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185292, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:52.468466 5485 init.cc:377] Logged in from keytab as oryx/127.30.146.190@KRBTEST.COM (short username oryx)
I20260808 10:34:52.469416 5485 webserver.cc:460] Webserver started at http://127.30.146.190:44127/ using document root <none> and password file <none>
I20260808 10:34:52.469941 5485 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:52.470005 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:52.470208 5485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:52.472215 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "602f69cad51b44c6ac6edf88980a20d7"
format_stamp: "Formatted at 2026-08-08 10:34:52 on dist-test-slave-9gcw"
server_key: "b9d0b587a5e5b786b9f7c9e3e03257ca"
server_key_iv: "da593867f31966d8809b174797d93a2d"
server_key_version: "encryptionkey@0"
I20260808 10:34:52.472812 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "602f69cad51b44c6ac6edf88980a20d7"
format_stamp: "Formatted at 2026-08-08 10:34:52 on dist-test-slave-9gcw"
server_key: "b9d0b587a5e5b786b9f7c9e3e03257ca"
server_key_iv: "da593867f31966d8809b174797d93a2d"
server_key_version: "encryptionkey@0"
I20260808 10:34:52.476753 5485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:34:52.479350 5500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:52.480826 5485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:52.480947 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "602f69cad51b44c6ac6edf88980a20d7"
format_stamp: "Formatted at 2026-08-08 10:34:52 on dist-test-slave-9gcw"
server_key: "b9d0b587a5e5b786b9f7c9e3e03257ca"
server_key_iv: "da593867f31966d8809b174797d93a2d"
server_key_version: "encryptionkey@0"
I20260808 10:34:52.481125 5485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:52.497802 5485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:52.501809 5485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:52.502051 5485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:52.510232 5485 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:34753
I20260808 10:34:52.510565 5554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:34753 every 8 connection(s)
I20260808 10:34:52.511451 5485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:52.513502 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 5485
I20260808 10:34:52.513789 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:52.514138 31306 external_mini_cluster.cc:1428] Setting key 93fa9fad8fcf9dac93dde3c9ca187de0
I20260808 10:34:52.514665 5555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:52.520860 5555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7: Bootstrap starting.
Aug 08 10:34:52 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:52.523104 5555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:52.523857 5555 log.cc:824] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:52.525838 5555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7: No bootstrap required, opened a new log
I20260808 10:34:52.528292 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:52.515715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.30.146.190:34753)
0808 10:34:52.516196 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:52.516210 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:52.516255 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:34:52.516919 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:52.516936 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:52.517371 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:52.517711 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:52.518570 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:52.519566 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:52.520451 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:52.520961 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:52.521174 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:52.523709 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:52.523714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:52.523727 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:52.523760 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:52.525531 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:52.525979 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:52.525992 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:52.525997 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:52.526105 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:52.526408 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:52.526412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:52.526414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:52.526809 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:52.527005 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:52.527314 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 10:34:52.527596 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":144,"threads_started":1}
I20260808 10:34:52.529681 5555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } }
I20260808 10:34:52.529953 5555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:52.530015 5555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602f69cad51b44c6ac6edf88980a20d7, State: Initialized, Role: FOLLOWER
I20260808 10:34:52.530488 5555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } }
I20260808 10:34:52.530648 5555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:52.530721 5555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:52.530789 5555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:52.531634 5555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } }
I20260808 10:34:52.531917 5555 leader_election.cc:302] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 602f69cad51b44c6ac6edf88980a20d7; no voters:
I20260808 10:34:52.532167 5555 leader_election.cc:288] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:52.532299 5562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:52.532565 5562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 1 LEADER]: Becoming Leader. State: Replica: 602f69cad51b44c6ac6edf88980a20d7, State: Running, Role: LEADER
I20260808 10:34:52.532876 5562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } }
I20260808 10:34:52.533440 5555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:52.534576 5565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 602f69cad51b44c6ac6edf88980a20d7. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } }
I20260808 10:34:52.534732 5565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:52.535035 5563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } }
I20260808 10:34:52.535147 5563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:52.535128 5572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:52.538612 5572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:52.544119 5572 catalog_manager.cc:1347] Generated new cluster ID: 6f9ec934f36e41308f35622429feb24c
I20260808 10:34:52.544196 5572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:52.573518 5572 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:52.574517 5572 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:52.579499 5572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7: Generated new TSK 0
I20260808 10:34:52.580210 5572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab.
Entry for principal oryx/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab.
I20260808 10:34:52.616901 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34753
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab
--principal=oryx/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:52.751106 5586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:52.751390 5586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:52.751435 5586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:52.755667 5586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:52.755745 5586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:52.755806 5586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:52.761267 5586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab
--principal=oryx/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:52.762392 5586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:52.763443 5586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:52.773564 5591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:52.773798 5586 server_base.cc:1034] running on GCE node
W20260808 10:34:52.773568 5593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:52.773564 5596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:52.774446 5586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:52.775343 5586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:52.776571 5586 hybrid_clock.cc:648] HybridClock initialized: now 1786185292776542 us; error 52 us; skew 500 ppm
Aug 08 10:34:52 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185292, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:52.780211 5586 init.cc:377] Logged in from keytab as oryx/127.30.146.129@KRBTEST.COM (short username oryx)
I20260808 10:34:52.781409 5586 webserver.cc:460] Webserver started at http://127.30.146.129:42883/ using document root <none> and password file <none>
I20260808 10:34:52.782076 5586 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:52.782194 5586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:52.782459 5586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:52.784907 5586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "83bc0009eb08456b9e74f477c73c7631"
format_stamp: "Formatted at 2026-08-08 10:34:52 on dist-test-slave-9gcw"
server_key: "1fcb9e0d85aa4322fcf64e467c41b1b3"
server_key_iv: "87ed87ce471699b6ff349fb50700e72a"
server_key_version: "encryptionkey@0"
I20260808 10:34:52.785786 5586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "83bc0009eb08456b9e74f477c73c7631"
format_stamp: "Formatted at 2026-08-08 10:34:52 on dist-test-slave-9gcw"
server_key: "1fcb9e0d85aa4322fcf64e467c41b1b3"
server_key_iv: "87ed87ce471699b6ff349fb50700e72a"
server_key_version: "encryptionkey@0"
I20260808 10:34:52.790344 5586 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:52.793138 5603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:52.794600 5586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:52.794749 5586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "83bc0009eb08456b9e74f477c73c7631"
format_stamp: "Formatted at 2026-08-08 10:34:52 on dist-test-slave-9gcw"
server_key: "1fcb9e0d85aa4322fcf64e467c41b1b3"
server_key_iv: "87ed87ce471699b6ff349fb50700e72a"
server_key_version: "encryptionkey@0"
I20260808 10:34:52.794876 5586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:52.811674 5586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:52.815491 5586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:52.815766 5586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:52.816494 5586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:52.817544 5586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:52.817633 5586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:52.817718 5586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:52.817780 5586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:52.828192 5586 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:36793
I20260808 10:34:52.828243 5724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:36793 every 8 connection(s)
I20260808 10:34:52.829282 5586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:52.835593 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 5586
I20260808 10:34:52.835734 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:52.836089 31306 external_mini_cluster.cc:1428] Setting key 35e1b427af806908d6dc646c566b9b99
Aug 08 10:34:52 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185292, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:52.846577 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:52.831281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:36863 (local address 127.30.146.190:34753)
0808 10:34:52.831443 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:52.831449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:52.832382 (+ 933us) server_negotiation.cc:407] Connection header received
0808 10:34:52.834086 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:52.834090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:52.834192 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:52.834274 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:52.835920 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:52.836597 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:52.838191 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:52.838446 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:52.838531 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:52.841896 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:52.841902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:52.841909 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:52.841945 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:52.843888 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:52.844539 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:52.844544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:52.844549 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:52.844656 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:52.845074 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:52.845080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:52.845083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:52.845415 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:52.845531 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:52.846206 (+ 675us) server_negotiation.cc:299] Negotiation successful
0808 10:34:52.846364 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:34:52.847462 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:52.831718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.129:36863)
0808 10:34:52.832211 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:52.832256 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:52.833810 (+ 1554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:52.834479 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:52.834492 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:52.834988 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:52.835697 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:52.835723 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:52.836779 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:52.836782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:52.838046 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:52.838057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:52.838623 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:52.838629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:52.838893 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:52.839572 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:52.839601 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:52.841691 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:52.844042 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:52.844047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:52.844062 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:52.844401 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:52.844786 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:52.844789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:52.844791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:52.844950 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:52.845566 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:52.845573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:52.845902 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 10:34:52.846087 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 10:34:52.846424 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for oryx/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:34:52.849064 5725 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34753
I20260808 10:34:52.849372 5725 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:52.849884 5725 heartbeater.cc:511] Master 127.30.146.190:34753 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.130@KRBTEST.COM" created.
I20260808 10:34:52.851867 5517 ts_manager.cc:194] Registered new tserver with Master: 83bc0009eb08456b9e74f477c73c7631 (127.30.146.129:36793)
I20260808 10:34:52.853602 5517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:36863
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab.
Entry for principal oryx/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab.
I20260808 10:34:52.873000 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34753
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab
--principal=oryx/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:53.007122 5730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:53.007479 5730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:53.007565 5730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:53.012351 5730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:53.012459 5730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:53.012562 5730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:53.019004 5730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab
--principal=oryx/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:53.020321 5730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:53.021574 5730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:34:53.032348 5730 server_base.cc:1034] running on GCE node
W20260808 10:34:53.032289 5737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:53.032285 5738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:53.032305 5740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:53.033128 5730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:53.034096 5730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:53.035332 5730 hybrid_clock.cc:648] HybridClock initialized: now 1786185293035305 us; error 37 us; skew 500 ppm
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:53.038494 5730 init.cc:377] Logged in from keytab as oryx/127.30.146.130@KRBTEST.COM (short username oryx)
I20260808 10:34:53.039456 5730 webserver.cc:460] Webserver started at http://127.30.146.130:42375/ using document root <none> and password file <none>
I20260808 10:34:53.040047 5730 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:53.040117 5730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:53.040310 5730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:53.042492 5730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "285a49e71818492aa9432d21c981af40"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "dcd2bedea8cb580bd214a5e486438d52"
server_key_iv: "001eed1c324a940fcf9e731c1f1b31e5"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.043042 5730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "285a49e71818492aa9432d21c981af40"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "dcd2bedea8cb580bd214a5e486438d52"
server_key_iv: "001eed1c324a940fcf9e731c1f1b31e5"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.047029 5730 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:34:53.049667 5747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:53.051034 5730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:53.051144 5730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "285a49e71818492aa9432d21c981af40"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "dcd2bedea8cb580bd214a5e486438d52"
server_key_iv: "001eed1c324a940fcf9e731c1f1b31e5"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.051246 5730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:53.078083 5730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:53.081966 5730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:53.082144 5730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:53.082716 5730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:53.083679 5730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:53.083729 5730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:53.083775 5730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:53.083791 5730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:53.097095 5730 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:41931
I20260808 10:34:53.097292 5869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:41931 every 8 connection(s)
I20260808 10:34:53.099213 5730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:53.102373 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 5730
I20260808 10:34:53.102502 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:53.102865 31306 external_mini_cluster.cc:1428] Setting key f6f894f482e17221f83e8fceac69a778
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.30.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.131@KRBTEST.COM" created.
I20260808 10:34:53.116986 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.101330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:48539 (local address 127.30.146.190:34753)
0808 10:34:53.101496 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.101501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.102668 (+ 1167us) server_negotiation.cc:407] Connection header received
0808 10:34:53.104542 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.104545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.104603 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.104702 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.106452 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.107144 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.108570 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.108838 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.108968 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.112225 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.112229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.112231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.112259 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.114217 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.115077 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.115080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.115081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.115226 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.115660 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.115668 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.115669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.115869 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.116016 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.116612 (+ 596us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.116804 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:34:53.118001 5872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.101784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.130:48539)
0808 10:34:53.102485 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.102550 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.104300 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.104884 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.104895 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.105482 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.106251 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.106275 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.107320 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.107324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.108398 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.108409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.109023 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.109028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.109314 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.110054 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.110073 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.111995 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.114394 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.114401 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.114426 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.114946 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.115353 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.115356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.115358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.115540 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.116022 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.116032 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.116350 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.116549 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.116930 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":115,"threads_started":1}
I20260808 10:34:53.119597 5870 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34753
I20260808 10:34:53.119879 5870 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:53.120415 5870 heartbeater.cc:511] Master 127.30.146.190:34753 requested a full tablet report, sending...
I20260808 10:34:53.121555 5517 ts_manager.cc:194] Registered new tserver with Master: 285a49e71818492aa9432d21c981af40 (127.30.146.130:41931)
I20260808 10:34:53.122181 5517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:48539
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab.
Entry for principal oryx/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab.
I20260808 10:34:53.138002 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34753
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab
--principal=oryx/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:53.256026 5875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:53.256361 5875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:53.256414 5875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:53.260628 5875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:53.260700 5875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:53.260761 5875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:53.266665 5875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab
--principal=oryx/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:53.267783 5875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:53.268862 5875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:34:53.279528 5875 server_base.cc:1034] running on GCE node
W20260808 10:34:53.279460 5884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:53.279471 5886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:53.279480 5883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:53.280237 5875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:53.281014 5875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:53.282223 5875 hybrid_clock.cc:648] HybridClock initialized: now 1786185293282202 us; error 34 us; skew 500 ppm
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:53.285415 5875 init.cc:377] Logged in from keytab as oryx/127.30.146.131@KRBTEST.COM (short username oryx)
I20260808 10:34:53.286430 5875 webserver.cc:460] Webserver started at http://127.30.146.131:44271/ using document root <none> and password file <none>
I20260808 10:34:53.287040 5875 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:53.287110 5875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:53.287330 5875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:53.289857 5875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "729af0f3753842848716e84689418f4c"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "431426f1bb5cbe25a7a925503dc8c1ec"
server_key_iv: "9d3f109dc21841bf52405b54f435790e"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.290449 5875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "729af0f3753842848716e84689418f4c"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "431426f1bb5cbe25a7a925503dc8c1ec"
server_key_iv: "9d3f109dc21841bf52405b54f435790e"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.294584 5875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:53.298354 5893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:53.299984 5875 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260808 10:34:53.300124 5875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "729af0f3753842848716e84689418f4c"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "431426f1bb5cbe25a7a925503dc8c1ec"
server_key_iv: "9d3f109dc21841bf52405b54f435790e"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.300256 5875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:53.309772 5875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:53.313287 5875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:53.313489 5875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:53.314055 5875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:53.314903 5875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:53.314950 5875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:53.314991 5875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:53.315006 5875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:53.324162 5875 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:36185
I20260808 10:34:53.324195 6016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:36185 every 8 connection(s)
I20260808 10:34:53.325063 5875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:53.326370 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 5875
I20260808 10:34:53.326534 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:53.326972 31306 external_mini_cluster.cc:1428] Setting key 693e0cdb9176940f8d830f7a17e2ebc6
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:53.339794 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.327075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:36031 (local address 127.30.146.190:34753)
0808 10:34:53.327214 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.327217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.328202 (+ 985us) server_negotiation.cc:407] Connection header received
0808 10:34:53.329805 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.329817 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.329875 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.329951 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.331265 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.331822 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.333009 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.333315 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.333432 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.336249 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.336253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.336256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.336292 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.337644 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.338197 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.338200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.338202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.338291 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.338643 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.338648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.338650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.338854 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.338966 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.339479 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.339611 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:34:53.340847 6019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.327511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.131:36031)
0808 10:34:53.328055 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.328109 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.329590 (+ 1481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.330125 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.330133 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.330619 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.331112 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.331126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.331964 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.331973 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.332873 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.332883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.333475 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.333479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.333720 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.334399 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.334415 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.336062 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.337772 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.337778 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.337793 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.338085 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.338390 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.338395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.338397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.338542 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.338965 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.338971 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.339229 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.339435 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.339776 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":161,"threads_started":1}
I20260808 10:34:53.342308 6017 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34753
I20260808 10:34:53.342695 6017 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:53.343259 6017 heartbeater.cc:511] Master 127.30.146.190:34753 requested a full tablet report, sending...
I20260808 10:34:53.344400 5517 ts_manager.cc:194] Registered new tserver with Master: 729af0f3753842848716e84689418f4c (127.30.146.131:36185)
I20260808 10:34:53.345242 5517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:36031
I20260808 10:34:53.353430 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.146.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.189.keytab.
Entry for principal oryx/127.30.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.189.keytab.
I20260808 10:34:53.384922 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.189:41421
--webserver_interface=127.30.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.189.keytab
--principal=oryx/127.30.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:34753,127.30.146.189:41421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:53.509509 6023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:53.509822 6023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:53.509869 6023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:53.514734 6023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:53.514814 6023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:53.514837 6023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:53.514855 6023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:53.514873 6023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:53.520953 6023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:34753,127.30.146.189:41421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.189.keytab
--principal=oryx/127.30.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.189:41421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
W20260808 10:34:53.521229 6023 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.146.190:34753,127.30.146.189:41421'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 10:34:53.522212 6023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:53.523329 6023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:53.533288 6028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:53.533407 6023 server_base.cc:1034] running on GCE node
W20260808 10:34:53.533416 6031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:53.533694 6029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:53.534406 6023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:53.535665 6023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:53.536862 6023 hybrid_clock.cc:648] HybridClock initialized: now 1786185293536830 us; error 47 us; skew 500 ppm
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:53.539913 6023 init.cc:377] Logged in from keytab as oryx/127.30.146.189@KRBTEST.COM (short username oryx)
I20260808 10:34:53.541038 6023 webserver.cc:460] Webserver started at http://127.30.146.189:40705/ using document root <none> and password file <none>
I20260808 10:34:53.541765 6023 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:53.541862 6023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:53.542142 6023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:53.544685 6023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data/instance:
uuid: "36e1691b6d774580985c7028e4901761"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "98c0c10e0491b10e0cf62b1e3d293e07"
server_key_iv: "b077cb5cb0ff029a32705f0e0423b6fb"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.545421 6023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal/instance:
uuid: "36e1691b6d774580985c7028e4901761"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "98c0c10e0491b10e0cf62b1e3d293e07"
server_key_iv: "b077cb5cb0ff029a32705f0e0423b6fb"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.549729 6023 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:34:53.552260 6039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:53.553579 6023 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:34:53.553728 6023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal
uuid: "36e1691b6d774580985c7028e4901761"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "98c0c10e0491b10e0cf62b1e3d293e07"
server_key_iv: "b077cb5cb0ff029a32705f0e0423b6fb"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.553848 6023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:53.574169 6023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:53.577709 6023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:53.577926 6023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:53.585350 6023 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.189:41421
I20260808 10:34:53.585726 6097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.189:41421 every 8 connection(s)
I20260808 10:34:53.586524 6023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
I20260808 10:34:53.589437 6098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:53.593009 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 6023
I20260808 10:34:53.593109 6098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.189@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.146.189@KRBTEST.COM
I20260808 10:34:53.609522 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.595216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.30.146.190:34753)
0808 10:34:53.595388 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.595393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.595732 (+ 339us) server_negotiation.cc:407] Connection header received
0808 10:34:53.596284 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.596288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.596340 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.596451 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.597754 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.598713 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.600318 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.600675 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.600790 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.604432 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.604436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.604438 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.604465 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.606784 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.607491 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.607495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.607497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.607614 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.608106 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.608109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.608111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.608430 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.608509 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.609182 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.609333 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:34:53.610598 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.594583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47662 (local address 127.30.146.189:41421)
0808 10:34:53.595127 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.595153 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.595189 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:34:53.596069 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.596089 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.596358 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.596541 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.597552 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.599053 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.600211 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.600828 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.601059 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.604980 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.604987 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.605026 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.605047 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.607369 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.607911 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.607917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.607919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.608006 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.608330 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.608333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.608334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.608897 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.609121 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.609391 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.609555 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":344,"thread_start_us":146,"threads_started":1}
I20260808 10:34:53.610603 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.595075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:41456)
0808 10:34:53.595501 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.595590 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.596079 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.596517 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.596522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.597071 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.597478 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.597502 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.598854 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.598857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.600177 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.600189 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.600896 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.600903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.601068 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.601974 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.602001 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.604192 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.606903 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.606911 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.606922 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.607374 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.607739 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.607742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.607744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.607963 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.608546 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.608553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.608898 (+ 345us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.609082 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.609508 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":48,"spinlock_wait_cycles":15360,"thread_start_us":138,"threads_started":1}
I20260808 10:34:53.612358 6098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.189@KRBTEST.COM for oryx/127.30.146.189@KRBTEST.COM
I20260808 10:34:53.620714 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.613071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:41421 (local address 127.0.0.1:47668)
0808 10:34:53.613250 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.613266 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.613354 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.613586 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.613589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.613800 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.613884 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.613890 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.614607 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.614612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.615135 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.615142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.615615 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.615618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.615782 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.616300 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.616322 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.617671 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.619347 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.619350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.619351 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.619544 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.619825 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.619828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.619830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.619920 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.620323 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.620326 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.620405 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.620491 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.620590 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":35}
I20260808 10:34:53.620841 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.613121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47668 (local address 127.30.146.189:41421)
0808 10:34:53.613251 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.613255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.613331 (+ 76us) server_negotiation.cc:407] Connection header received
0808 10:34:53.613447 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.613450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.613487 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.613577 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.613996 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.614503 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.615255 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.615507 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.615617 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.617808 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.617812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.617814 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.617836 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.619244 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.619633 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.619635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.619637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.619709 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.620013 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.620015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.620016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.620220 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.620303 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.620551 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.620661 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":72}
I20260808 10:34:53.624537 6098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: Bootstrap starting.
I20260808 10:34:53.627106 6098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:53.628053 6098 log.cc:824] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:53.630303 6098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: No bootstrap required, opened a new log
I20260808 10:34:53.633560 6098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } }
I20260808 10:34:53.633816 6098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:53.633900 6098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36e1691b6d774580985c7028e4901761, State: Initialized, Role: FOLLOWER
I20260808 10:34:53.634447 6098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } }
I20260808 10:34:53.635048 6104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } } }
I20260808 10:34:53.635175 6104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:34:53.635774 6098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 10:34:53.640740 6116 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:34:53.640848 6116 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:34:53.644757 6023 master_runner.cc:428] Detected that this master 36e1691b6d774580985c7028e4901761 is joining an existing cluster
I20260808 10:34:53.644850 6023 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.146.189:41421
I20260808 10:34:53.656497 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.647382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41470 (local address 127.30.146.190:34753)
0808 10:34:53.647520 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.647526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.647909 (+ 383us) server_negotiation.cc:407] Connection header received
0808 10:34:53.648107 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.648110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.648162 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.648249 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.648919 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.649835 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.650786 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.651051 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.651201 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.652584 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.652592 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.652599 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.652638 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.654313 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.654939 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.654943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.654945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.655036 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.655422 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.655427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.655431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.655724 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.655844 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.656238 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.656364 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:34:53.656518 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.647325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:41470)
0808 10:34:53.647812 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.647829 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.647977 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.648370 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.648373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.648649 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.648781 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.648789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.649939 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.649949 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.650640 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.650648 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.651219 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.651223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.651328 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.651961 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.651977 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.652449 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.654511 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.654514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.654517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.654842 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.655142 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.655145 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.655147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.655246 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.655845 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.655848 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.655959 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.656160 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.656310 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":105,"threads_started":1}
I20260808 10:34:53.659385 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.647392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:41421 (local address 127.0.0.1:47680)
0808 10:34:53.647822 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.647836 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.647946 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.648231 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.648233 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.648523 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.648654 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.648661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.649871 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.649875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.651487 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.651500 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.652165 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.652169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.652289 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.653015 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.653037 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.654759 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.657654 (+ 2895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.657669 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.657671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.657922 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.658229 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.658232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.658234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.658304 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.658789 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.658792 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.658887 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.658993 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.659121 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260808 10:34:53.659394 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.647752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47680 (local address 127.30.146.189:41421)
0808 10:34:53.647883 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.647890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.647906 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:34:53.648048 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.648054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.648113 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.648217 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.648779 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.649733 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.651670 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.652022 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.652163 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.654896 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.654900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.654904 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.654934 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.657531 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.658049 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.658052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.658054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.658124 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.658400 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.658404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.658406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.658676 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.658784 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.659072 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.659203 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":56}
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185292, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.190@KRBTEST.COM for oryx/127.30.146.189@KRBTEST.COM
I20260808 10:34:53.674221 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.662750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47684 (local address 127.30.146.189:41421)
0808 10:34:53.662881 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.662886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.663406 (+ 520us) server_negotiation.cc:407] Connection header received
0808 10:34:53.663623 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.663627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.663680 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.663754 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.664680 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.665510 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.666997 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.667258 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.667381 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.670220 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.670223 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.670224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.670247 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.671917 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.672459 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.672464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.672466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.672559 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.672994 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.672999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.673001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.673322 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.673454 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.673865 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.674014 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:34:53.674234 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.662849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:41421 (local address 127.0.0.1:47684)
0808 10:34:53.663260 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.663317 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.663465 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.663897 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.663904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.664378 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.664540 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.664549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.665649 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.665656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.666856 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.666883 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.667388 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.667392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.667536 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.668180 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.668194 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.670016 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.672048 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.672060 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.672067 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.672339 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.672784 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.672788 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.672790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.672883 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.673454 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.673462 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.673611 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.673802 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.674039 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for oryx/127.30.146.188@KRBTEST.COM; defaulting to no policy
I20260808 10:34:53.676273 5517 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.146.189:41421: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "36e1691b6d774580985c7028e4901761"
member_type: NON_VOTER
last_known_addr {
host: "127.30.146.189"
port: 41421
}
attrs {
promote: true
}
}
dest_uuid: "602f69cad51b44c6ac6edf88980a20d7"
cas_config_opid_index: -1
I20260808 10:34:53.677204 5517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.188@KRBTEST.COM" created.
I20260808 10:34:53.679328 6128 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 36e1691b6d774580985c7028e4901761 (127.30.146.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } } }
I20260808 10:34:53.680250 6069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:53.681713 6128 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.146.189:41421
I20260808 10:34:53.681939 6069 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 FOLLOWER]: Refusing update from remote peer 602f69cad51b44c6ac6edf88980a20d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 10:34:53.681994 6128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } } }
I20260808 10:34:53.682083 6128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:53.682345 6023 master.cc:537] Master@127.30.146.189:41421 shutting down...
I20260808 10:34:53.682778 6127 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:34:53.685506 5501 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 10:34:53.686658 5501 consensus_peers.cc:487] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 -> Peer 36e1691b6d774580985c7028e4901761 (127.30.146.189:41421): Couldn't send request to peer 36e1691b6d774580985c7028e4901761. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:34:53.687625 6023 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:53.687950 6023 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:53.688092 6023 tablet_replica.cc:330] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.188.keytab.
Entry for principal oryx/127.30.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.188.keytab.
I20260808 10:34:53.705698 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.188:34901
--webserver_interface=127.30.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.188.keytab
--principal=oryx/127.30.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:34753,127.30.146.189:41421,127.30.146.188:34901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:34:53.704845 6023 master.cc:552] Master@127.30.146.189:41421 shutdown complete.
I20260808 10:34:53.707576 6023 master_runner.cc:305] Clearing existing system tablet
I20260808 10:34:53.707911 6023 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:53.707979 6023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:53.709607 6023 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:34:53.710528 6135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:53.710755 6023 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:53.710891 6023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:53.710934 6023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal
uuid: "36e1691b6d774580985c7028e4901761"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "98c0c10e0491b10e0cf62b1e3d293e07"
server_key_iv: "b077cb5cb0ff029a32705f0e0423b6fb"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.712769 6023 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:53.714931 6023 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 10:34:53.715047 6023 log.cc:1196] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 10:34:53.715301 6023 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: Deleting consensus metadata
I20260808 10:34:53.715518 6023 master_runner.cc:315] Copying system tablet from 127.30.146.190:34753
I20260808 10:34:53.716872 6023 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: tablet copy: Beginning tablet copy session from remote peer at address 127.30.146.190:34753
I20260808 10:34:53.725421 6141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.717573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:41474)
0808 10:34:53.717982 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.717995 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.718144 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.718441 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.718444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.718681 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.718779 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.718785 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.719604 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.719609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.720217 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.720225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.720717 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.720720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.720821 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.721495 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.721508 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.721867 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.723744 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.723747 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.723750 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.723965 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.724269 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.724273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.724275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.724366 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.724778 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.724781 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.724895 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.725007 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.725191 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":99,"threads_started":1}
I20260808 10:34:53.725411 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.717691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41474 (local address 127.30.146.190:34753)
0808 10:34:53.717840 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.717845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.718092 (+ 247us) server_negotiation.cc:407] Connection header received
0808 10:34:53.718271 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.718274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.718331 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.718422 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.718905 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.719483 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.720345 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.720594 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.720711 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.722023 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.722027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.722030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.722057 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.723604 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.724070 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.724074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.724076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.724172 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.724471 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.724474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.724476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.724676 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.724791 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.725067 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.725240 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:34:53.726434 5539 tablet_copy_service.cc:140] P 602f69cad51b44c6ac6edf88980a20d7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 36e1691b6d774580985c7028e4901761 ({username='oryx', principal='oryx/127.30.146.189@KRBTEST.COM'} at 127.0.0.1:41474)
I20260808 10:34:53.726645 5539 tablet_copy_service.cc:161] P 602f69cad51b44c6ac6edf88980a20d7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 36e1691b6d774580985c7028e4901761 at {username='oryx', principal='oryx/127.30.146.189@KRBTEST.COM'} at 127.0.0.1:41474: session id = 36e1691b6d774580985c7028e4901761-00000000000000000000000000000000
I20260808 10:34:53.728423 5539 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:34:53.730311 6023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 10:34:53.732442 6023 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: tablet copy: Starting download of 0 data blocks...
I20260808 10:34:53.732698 6023 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: tablet copy: Starting download of 1 WAL segments...
I20260808 10:34:53.734659 6023 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:34:53.736451 5539 tablet_copy_service.cc:342] P 602f69cad51b44c6ac6edf88980a20d7: Request end of tablet copy session 36e1691b6d774580985c7028e4901761-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.146.189@KRBTEST.COM'} at 127.0.0.1:41474
I20260808 10:34:53.736647 5539 tablet_copy_service.cc:433] P 602f69cad51b44c6ac6edf88980a20d7: ending tablet copy session 36e1691b6d774580985c7028e4901761-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 36e1691b6d774580985c7028e4901761
I20260808 10:34:53.743922 6023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:53.745790 6145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:53.745982 6142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:53.746155 6143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:53.746323 6023 server_base.cc:1034] running on GCE node
I20260808 10:34:53.746479 6023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:53.746797 6023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:53.747939 6023 hybrid_clock.cc:648] HybridClock initialized: now 1786185293747930 us; error 26 us; skew 500 ppm
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:53.749570 6023 init.cc:377] Logged in from keytab as oryx/127.30.146.189@KRBTEST.COM (short username oryx)
I20260808 10:34:53.750190 6023 webserver.cc:460] Webserver started at http://127.30.146.189:37031/ using document root <none> and password file <none>
I20260808 10:34:53.750398 6023 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:53.750455 6023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:53.751637 6023 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:53.752569 6152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:53.752840 6023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:53.752938 6023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal
uuid: "36e1691b6d774580985c7028e4901761"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "98c0c10e0491b10e0cf62b1e3d293e07"
server_key_iv: "b077cb5cb0ff029a32705f0e0423b6fb"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.753046 6023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:53.774725 6023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:53.776818 6023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:53.781980 6023 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.189:41421
I20260808 10:34:53.782246 6214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.189:41421 every 8 connection(s)
I20260808 10:34:53.782653 6023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
I20260808 10:34:53.784243 6215 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:34:53.785295 6215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: Bootstrap starting.
I20260808 10:34:53.793915 6215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:34:53.794327 6215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: Bootstrap complete.
I20260808 10:34:53.794914 6215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } }
I20260808 10:34:53.795051 6215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 36e1691b6d774580985c7028e4901761, State: Initialized, Role: LEARNER
I20260808 10:34:53.795282 6215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } }
I20260808 10:34:53.795733 6218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } } }
I20260808 10:34:53.795869 6218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: This master's current role is: LEARNER
I20260808 10:34:53.795953 6215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:53.799302 6232 catalog_manager.cc:1259] Loaded cluster ID: 6f9ec934f36e41308f35622429feb24c
I20260808 10:34:53.799371 6232 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: loading cluster ID for follower catalog manager: success
I20260808 10:34:53.801021 6232 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: acquiring CA information for follower catalog manager: success
I20260808 10:34:53.803164 6232 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:34:53.856295 5725 heartbeater.cc:503] Master 127.30.146.190:34753 was elected leader, sending a full tablet report...
W20260808 10:34:53.867275 6133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:53.867620 6133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:53.867712 6133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:53.871884 6133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:53.871980 6133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:53.872020 6133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:53.872079 6133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:53.872143 6133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:53.877763 6133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:34753,127.30.146.189:41421,127.30.146.188:34901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.188.keytab
--principal=oryx/127.30.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.188:34901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:53.878952 6133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:53.880025 6133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:53.888814 6237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:53.888823 6241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:53.888823 6239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:53.889209 6133 server_base.cc:1034] running on GCE node
I20260808 10:34:53.889717 6133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:53.890803 6133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:53.891973 6133 hybrid_clock.cc:648] HybridClock initialized: now 1786185293891954 us; error 33 us; skew 500 ppm
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:53.894860 6133 init.cc:377] Logged in from keytab as oryx/127.30.146.188@KRBTEST.COM (short username oryx)
I20260808 10:34:53.895761 6133 webserver.cc:460] Webserver started at http://127.30.146.188:38749/ using document root <none> and password file <none>
I20260808 10:34:53.896346 6133 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:53.896425 6133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:53.896658 6133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:53.899420 6133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data/instance:
uuid: "9089e96260c7419b8e9c439d15ecb67a"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "d8bd522416586a3f324ac4cc49b3f0f4"
server_key_iv: "410a6ea438fd003aa268f68e711a0b4d"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.900017 6133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal/instance:
uuid: "9089e96260c7419b8e9c439d15ecb67a"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "d8bd522416586a3f324ac4cc49b3f0f4"
server_key_iv: "410a6ea438fd003aa268f68e711a0b4d"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.904259 6133 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 10:34:53.906991 6249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:53.908353 6133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:53.908494 6133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal
uuid: "9089e96260c7419b8e9c439d15ecb67a"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "d8bd522416586a3f324ac4cc49b3f0f4"
server_key_iv: "410a6ea438fd003aa268f68e711a0b4d"
server_key_version: "encryptionkey@0"
I20260808 10:34:53.908612 6133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:53.936564 6133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:53.940062 6133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:53.940286 6133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:53.947623 6133 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.188:34901
I20260808 10:34:53.947930 6308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.188:34901 every 8 connection(s)
I20260808 10:34:53.948679 6133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
I20260808 10:34:53.951644 6309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:53.955003 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 6133
I20260808 10:34:53.955806 6309 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.188@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.146.188@KRBTEST.COM
I20260808 10:34:53.969980 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.958008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41480 (local address 127.30.146.190:34753)
0808 10:34:53.958153 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.958157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.958437 (+ 280us) server_negotiation.cc:407] Connection header received
0808 10:34:53.958748 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.958751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.958801 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.958881 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.960039 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.960860 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.962369 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.962721 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.962833 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.965980 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.965985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.965990 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.966029 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.967602 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.968307 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.968311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.968313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.968415 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.968765 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.968767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.968769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.969027 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.969135 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.969740 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.969852 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:34:53.970788 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.957823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:41480)
0808 10:34:53.958287 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.958338 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.958555 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.958959 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.958991 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.959485 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.959900 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.959913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.960996 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.960998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.962202 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.962213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.962872 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.962875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.963056 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.963772 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.963813 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.965807 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.967827 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.967836 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.967847 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.968197 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.968507 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.968509 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.968510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.968645 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.969136 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.969161 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.969443 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.969611 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.969869 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":52,"thread_start_us":198,"threads_started":1}
I20260808 10:34:53.972276 6309 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:34:53.972432 6311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.956833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41084 (local address 127.30.146.188:34901)
0808 10:34:53.957313 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.957331 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.957371 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:34:53.958193 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.958215 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.958557 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.958799 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.959795 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.961135 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.962364 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.962936 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.963053 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.966397 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.966405 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.966420 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.966447 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.968274 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.968743 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.968749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.968751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.968836 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.971160 (+ 2324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.971163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.971164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.971512 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.971734 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.971986 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.972161 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":331,"thread_start_us":138,"threads_started":1}
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.188@KRBTEST.COM for oryx/127.30.146.189@KRBTEST.COM
I20260808 10:34:53.981843 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.973414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:41421 (local address 127.0.0.1:47696)
0808 10:34:53.973567 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.973581 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.973649 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.974058 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.974061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.974231 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.974358 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.974367 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.975294 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.975299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.975878 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.975885 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.976390 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.976394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.976491 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.977027 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.977040 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.978481 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.980204 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.980207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.980208 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.980431 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.980880 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.980882 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.980884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.980964 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.981429 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.981431 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.981511 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.981594 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.981699 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 10:34:53.982097 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.973531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47696 (local address 127.30.146.189:41421)
0808 10:34:53.973851 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.973854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.973867 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:34:53.973904 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.973906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.973943 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.974018 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.974474 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.975158 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.976008 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.976265 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.976380 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.978616 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.978619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.978621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.978702 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.980057 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.980552 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.980556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.980558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.980747 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.981054 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.981056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.981058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.981338 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.981442 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.981739 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.981853 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":175,"threads_started":1}
I20260808 10:34:53.982860 6309 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:34:53 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185293, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.188@KRBTEST.COM for oryx/127.30.146.188@KRBTEST.COM
I20260808 10:34:53.990701 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.983305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:34901 (local address 127.0.0.1:41090)
0808 10:34:53.983433 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:53.983450 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:53.983532 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:53.983741 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:53.983743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:53.983888 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:53.983964 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.983969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.984677 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.984681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.985328 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:53.985337 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.985774 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.985777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:53.985841 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.986215 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:53.986225 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:53.987514 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:53.989259 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.989262 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.989264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.989450 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.989732 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:53.989735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:53.989737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:53.989820 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:53.990178 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:53.990181 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:53.990275 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 10:34:53.990377 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 10:34:53.990514 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":1}
I20260808 10:34:53.990701 6311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:53.983391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.30.146.188:34901)
0808 10:34:53.983503 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:53.983507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:53.983521 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:34:53.983613 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:53.983615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:53.983647 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:53.983708 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:53.984060 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.984542 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.985431 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:53.985672 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:53.985773 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:53.987653 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:53.987656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:53.987658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:53.987676 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:53.989091 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.989554 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.989559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.989561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.989635 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:53.989906 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:53.989910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:53.989912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:53.990077 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:53.990178 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:53.990420 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 10:34:53.990566 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":43}
I20260808 10:34:53.994452 6309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Bootstrap starting.
I20260808 10:34:53.996965 6309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:53.997855 6309 log.cc:824] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:54.000211 6309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: No bootstrap required, opened a new log
I20260808 10:34:54.003367 6309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } }
I20260808 10:34:54.003607 6309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:54.003643 6309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9089e96260c7419b8e9c439d15ecb67a, State: Initialized, Role: FOLLOWER
I20260808 10:34:54.004153 6309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } }
I20260808 10:34:54.004843 6315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } } }
I20260808 10:34:54.004994 6315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:34:54.005722 6309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: configured and running, proceeding with master startup.
W20260808 10:34:54.011008 6328 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:34:54.011116 6328 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:34:54.014642 6133 master_runner.cc:428] Detected that this master 9089e96260c7419b8e9c439d15ecb67a is joining an existing cluster
I20260808 10:34:54.014735 6133 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.146.188:34901
I20260808 10:34:54.025866 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 5586
I20260808 10:34:54.026276 6311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.017467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.30.146.188:34901)
0808 10:34:54.017567 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.017572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.018054 (+ 482us) server_negotiation.cc:407] Connection header received
0808 10:34:54.018225 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.018228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.018269 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.018313 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.019079 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.019834 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.020971 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.021325 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.021401 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.022895 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.022898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.022900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.022919 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.024457 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.024969 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.024975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.024980 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.025074 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.025407 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.025410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.025412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.025639 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.025750 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.026029 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.026139 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:34:54.026808 6336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.017656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:34901 (local address 127.0.0.1:41092)
0808 10:34:54.017977 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.017991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.018131 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.018449 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.018452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.018764 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.018958 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.018966 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.019957 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.019961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.020846 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.020856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.021561 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.021564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.021670 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.022235 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.022250 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.022760 (+ 510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.024594 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.024598 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.024600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.024862 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.025198 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.025201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.025204 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.025300 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.025749 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.025752 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.025851 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.026452 (+ 601us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.026597 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":23,"spinlock_wait_cycles":15616,"thread_start_us":88,"threads_started":1}
I20260808 10:34:54.037974 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:36793
--local_ip_for_outbound_sockets=127.30.146.129
--tserver_master_addrs=127.30.146.190:34753
--webserver_port=42883
--webserver_interface=127.30.146.129
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab
--principal=oryx/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.146.190:34753,127.30.146.189:41421
--tserver_master_addrs=127.30.146.190:34753,127.30.146.189:41421,127.30.146.188:34901 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:34:54.042892 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.016955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41484 (local address 127.30.146.190:34753)
0808 10:34:54.017101 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.017106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.017326 (+ 220us) server_negotiation.cc:407] Connection header received
0808 10:34:54.017539 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.017546 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.017596 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.017651 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.018414 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.019247 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.021179 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.021523 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.021599 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.038710 (+ 17111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.038715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.038719 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.038752 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.040716 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.041343 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.041347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.041361 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.041463 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.041781 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.041787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.041792 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.042064 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.042148 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.042546 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.042692 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:34:54.043215 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.017395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47700 (local address 127.30.146.189:41421)
0808 10:34:54.017527 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.017531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.018047 (+ 516us) server_negotiation.cc:407] Connection header received
0808 10:34:54.018235 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.018238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.018288 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.018343 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.019079 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.019833 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.021179 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.021523 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.021599 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.038949 (+ 17350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.038956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.038959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.038998 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.041346 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.041857 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.041860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.041862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.041939 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.042245 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.042249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.042251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.042480 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.042549 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.042928 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.043064 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:34:54.043550 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.016875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:41484)
0808 10:34:54.017228 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.017254 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.017416 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.017824 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.017827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.018090 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.018281 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.018288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.019380 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.019383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.021011 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.021019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.021755 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.021759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.021933 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.037829 (+ 15896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.037854 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.038467 (+ 613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.040894 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.040899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.040902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.041198 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.041578 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.041581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.041583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.041664 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.042206 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.042211 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.042352 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.043134 (+ 782us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.043309 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":53,"spinlock_wait_cycles":2816,"thread_start_us":283,"threads_started":1}
I20260808 10:34:54.044312 6337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.016905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:41421 (local address 127.0.0.1:47700)
0808 10:34:54.017962 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.017976 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.018124 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.018488 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.018490 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.018769 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.018945 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.018952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.020034 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.020038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.021008 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.021019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.021755 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.021759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.022242 (+ 483us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.037985 (+ 15743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.038005 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.038752 (+ 747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.041486 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.041492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.041497 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.041746 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.042056 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.042059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.042061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.042143 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.042622 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.042624 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.042730 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.043985 (+ 1255us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.044134 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":386,"spinlock_wait_cycles":11776,"thread_start_us":109,"threads_started":1}
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185292, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.190@KRBTEST.COM for oryx/127.30.146.188@KRBTEST.COM
I20260808 10:34:54.055953 6311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.047627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.30.146.188:34901)
0808 10:34:54.047734 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.047738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.047815 (+ 77us) server_negotiation.cc:407] Connection header received
0808 10:34:54.047983 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.047988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.048038 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.048125 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.048539 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.049042 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.049950 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.050180 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.050298 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.052296 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.052300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.052303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.052326 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.054061 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.054596 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.054599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.054601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.054689 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.055009 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.055014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.055015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.055197 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.055309 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.055598 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.055750 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:34:54.055986 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.047581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:34901 (local address 127.0.0.1:41104)
0808 10:34:54.047757 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.047772 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.047858 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.048143 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.048145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.048337 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.048426 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.048431 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.049249 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.049255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.049825 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.049834 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.050299 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.050302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.050398 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.050864 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.050878 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.052154 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.054174 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.054178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.054181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.054420 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.054804 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.054807 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.054809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.054928 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.055303 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.055307 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.055428 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.055543 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.055835 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 10:34:54.057322 5517 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.146.188:34901: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a"
member_type: NON_VOTER
last_known_addr {
host: "127.30.146.188"
port: 34901
}
attrs {
promote: true
}
}
dest_uuid: "602f69cad51b44c6ac6edf88980a20d7"
cas_config_opid_index: 5
I20260808 10:34:54.057930 5517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } }
I20260808 10:34:54.059104 6127 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9089e96260c7419b8e9c439d15ecb67a (127.30.146.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.060451 6127 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.146.188:34901
I20260808 10:34:54.060622 6127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.060745 6127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:54.061602 6133 master.cc:537] Master@127.30.146.188:34901 shutting down...
I20260808 10:34:54.061805 6280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:54.063393 6280 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 FOLLOWER]: Refusing update from remote peer 602f69cad51b44c6ac6edf88980a20d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 10:34:54.064481 6128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:34:54.066883 5504 consensus_peers.cc:487] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 -> Peer 9089e96260c7419b8e9c439d15ecb67a (127.30.146.188:34901): Couldn't send request to peer 9089e96260c7419b8e9c439d15ecb67a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:34:54.069586 6133 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:54.069950 6133 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:54.070130 6133 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: stopping tablet replica
I20260808 10:34:54.071067 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.059235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:41421 (local address 127.0.0.1:47716)
0808 10:34:54.062348 (+ 3113us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.062362 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.062494 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.062930 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.062933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.062950 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:54.063055 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.063065 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.068042 (+ 4977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.068045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.069356 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.069367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.070596 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.070601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.070698 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.070747 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.070844 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.070938 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3029}
I20260808 10:34:54.071453 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.060978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47716 (local address 127.30.146.189:41421)
0808 10:34:54.061392 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.061397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.062422 (+ 1025us) server_negotiation.cc:407] Connection header received
0808 10:34:54.062612 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.062616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.062681 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.062754 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:54.063198 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.064050 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.069516 (+ 5466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.070227 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.070345 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.070490 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.071251 (+ 761us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.071341 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317}
I20260808 10:34:54.071910 6187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 LEARNER]: Refusing update from remote peer 602f69cad51b44c6ac6edf88980a20d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 10:34:54.072381 6128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 10:34:54.074304 6187 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9089e96260c7419b8e9c439d15ecb67a (127.30.146.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.076522 6218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 602f69cad51b44c6ac6edf88980a20d7. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.076646 6218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: This master's current role is: LEARNER
I20260808 10:34:54.076756 6218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: NON_VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: true } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.076840 6218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: This master's current role is: LEARNER
I20260808 10:34:54.078604 6128 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7: attempting to promote NON_VOTER 36e1691b6d774580985c7028e4901761 to VOTER
I20260808 10:34:54.079123 6128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } }
I20260808 10:34:54.080353 6187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 LEARNER]: Refusing update from remote peer 602f69cad51b44c6ac6edf88980a20d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 10:34:54.080480 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.080120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:34901 (local address 127.0.0.1:41106)
0808 10:34:54.080266 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.080338 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:34901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 10:34:54.080762 6127 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 10:34:54.081529 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.081108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:34901 (local address 127.0.0.1:41112)
0808 10:34:54.081404 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.081454 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:34901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":126,"threads_started":1}
W20260808 10:34:54.081684 5504 consensus_peers.cc:487] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 -> Peer 9089e96260c7419b8e9c439d15ecb67a (127.30.146.188:34901): Couldn't send request to peer 9089e96260c7419b8e9c439d15ecb67a. Status: Network error: Client connection negotiation failed: client connection to 127.30.146.188:34901: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 10:34:54.082484 6130 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 36e1691b6d774580985c7028e4901761 (127.30.146.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.082957 6187 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 36e1691b6d774580985c7028e4901761 (127.30.146.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.084133 6130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.084240 6130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:54.084367 6130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.084452 6130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:54.085908 6340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.086066 6340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:34:54.090809 6133 master.cc:552] Master@127.30.146.188:34901 shutdown complete.
I20260808 10:34:54.090881 6133 master_runner.cc:305] Clearing existing system tablet
I20260808 10:34:54.091122 6133 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:54.091203 6133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:54.092384 6133 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:34:54.093729 6343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.094183 6133 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:54.094416 6133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:54.094455 6133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal
uuid: "9089e96260c7419b8e9c439d15ecb67a"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "d8bd522416586a3f324ac4cc49b3f0f4"
server_key_iv: "410a6ea438fd003aa268f68e711a0b4d"
server_key_version: "encryptionkey@0"
I20260808 10:34:54.096680 6133 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:54.099617 6133 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 10:34:54.099709 6133 log.cc:1196] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 10:34:54.099970 6133 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Deleting consensus metadata
I20260808 10:34:54.100121 6133 master_runner.cc:315] Copying system tablet from 127.30.146.190:34753
I20260808 10:34:54.101598 6133 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: tablet copy: Beginning tablet copy session from remote peer at address 127.30.146.190:34753
I20260808 10:34:54.111627 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.103333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41498 (local address 127.30.146.190:34753)
0808 10:34:54.103478 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.103483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.103721 (+ 238us) server_negotiation.cc:407] Connection header received
0808 10:34:54.103897 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.103900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.103961 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.104053 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.104525 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.105216 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.106270 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.106577 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.106680 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.108165 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.108171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.108178 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.108204 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.109706 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.110180 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.110183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.110184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.110263 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.110549 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.110554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.110556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.110754 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.110868 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.111178 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.111381 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:34:54.112427 6350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.103258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:41498)
0808 10:34:54.103612 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.103625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.103779 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.104070 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.104073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.104288 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.104393 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.104399 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.105332 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.105336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.106144 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.106153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.106871 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.106875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.107017 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.107595 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.107612 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.108018 (+ 406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.109836 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.109840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.109843 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.110059 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.110359 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.110362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.110364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.110457 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.110855 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.110858 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.110967 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.112116 (+ 1149us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.112267 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":91,"threads_started":1}
I20260808 10:34:54.113245 5539 tablet_copy_service.cc:140] P 602f69cad51b44c6ac6edf88980a20d7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9089e96260c7419b8e9c439d15ecb67a ({username='oryx', principal='oryx/127.30.146.188@KRBTEST.COM'} at 127.0.0.1:41498)
I20260808 10:34:54.113387 5539 tablet_copy_service.cc:161] P 602f69cad51b44c6ac6edf88980a20d7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9089e96260c7419b8e9c439d15ecb67a at {username='oryx', principal='oryx/127.30.146.188@KRBTEST.COM'} at 127.0.0.1:41498: session id = 9089e96260c7419b8e9c439d15ecb67a-00000000000000000000000000000000
I20260808 10:34:54.114388 5539 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:34:54.115933 6133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 10:34:54.118316 6133 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: tablet copy: Starting download of 0 data blocks...
I20260808 10:34:54.118553 6133 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: tablet copy: Starting download of 1 WAL segments...
I20260808 10:34:54.120287 6133 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:34:54.122167 5539 tablet_copy_service.cc:342] P 602f69cad51b44c6ac6edf88980a20d7: Request end of tablet copy session 9089e96260c7419b8e9c439d15ecb67a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.146.188@KRBTEST.COM'} at 127.0.0.1:41498
I20260808 10:34:54.122287 5539 tablet_copy_service.cc:433] P 602f69cad51b44c6ac6edf88980a20d7: ending tablet copy session 9089e96260c7419b8e9c439d15ecb67a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9089e96260c7419b8e9c439d15ecb67a
I20260808 10:34:54.124402 5870 heartbeater.cc:503] Master 127.30.146.190:34753 was elected leader, sending a full tablet report...
I20260808 10:34:54.129779 6133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:54.131618 6355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:54.131863 6351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:54.137339 6352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:54.137347 6133 server_base.cc:1034] running on GCE node
I20260808 10:34:54.137633 6133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:54.137982 6133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:54.139125 6133 hybrid_clock.cc:648] HybridClock initialized: now 1786185294139126 us; error 54 us; skew 500 ppm
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:54.140923 6133 init.cc:377] Logged in from keytab as oryx/127.30.146.188@KRBTEST.COM (short username oryx)
I20260808 10:34:54.141607 6133 webserver.cc:460] Webserver started at http://127.30.146.188:39225/ using document root <none> and password file <none>
I20260808 10:34:54.141846 6133 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:54.141914 6133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:54.143422 6133 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:34:54.144585 6364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.144867 6133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:54.144981 6133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal
uuid: "9089e96260c7419b8e9c439d15ecb67a"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "d8bd522416586a3f324ac4cc49b3f0f4"
server_key_iv: "410a6ea438fd003aa268f68e711a0b4d"
server_key_version: "encryptionkey@0"
I20260808 10:34:54.145107 6133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:54.160144 6133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:54.162657 6133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:54.168700 6133 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.188:34901
I20260808 10:34:54.169289 6424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.188:34901 every 8 connection(s)
I20260808 10:34:54.169557 6133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
I20260808 10:34:54.171228 6425 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:34:54.172156 6425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Bootstrap starting.
I20260808 10:34:54.182870 6425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:34:54.183228 6425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Bootstrap complete.
I20260808 10:34:54.183835 6425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } }
I20260808 10:34:54.183975 6425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9089e96260c7419b8e9c439d15ecb67a, State: Initialized, Role: LEARNER
I20260808 10:34:54.184093 6425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } }
I20260808 10:34:54.184553 6425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:54.185038 6428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.185369 6428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: This master's current role is: LEARNER
I20260808 10:34:54.188199 6439 catalog_manager.cc:1259] Loaded cluster ID: 6f9ec934f36e41308f35622429feb24c
I20260808 10:34:54.188294 6439 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: loading cluster ID for follower catalog manager: success
I20260808 10:34:54.190107 6439 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: acquiring CA information for follower catalog manager: success
I20260808 10:34:54.191249 6439 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 10:34:54.213558 6338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:54.213915 6338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:54.213977 6338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:54.220327 6338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:54.220431 6338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:54.220522 6338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:54.229565 6338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab
--principal=oryx/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:36793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=42883
--tserver_master_addrs=127.30.146.190:34753,127.30.146.189:41421,127.30.146.188:34901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:54.235411 6338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:54.236846 6338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:54.250883 6445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:54.250933 6448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:54.251689 6444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:54.252019 6338 server_base.cc:1034] running on GCE node
I20260808 10:34:54.252453 6338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:54.253315 6338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:54.254482 6338 hybrid_clock.cc:648] HybridClock initialized: now 1786185294254445 us; error 57 us; skew 500 ppm
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:54.257602 6338 init.cc:377] Logged in from keytab as oryx/127.30.146.129@KRBTEST.COM (short username oryx)
I20260808 10:34:54.258683 6338 webserver.cc:460] Webserver started at http://127.30.146.129:42883/ using document root <none> and password file <none>
I20260808 10:34:54.259294 6338 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:54.259368 6338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:54.263411 6338 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 10:34:54.265702 6455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.267088 6338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:54.267189 6338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "83bc0009eb08456b9e74f477c73c7631"
format_stamp: "Formatted at 2026-08-08 10:34:52 on dist-test-slave-9gcw"
server_key: "1fcb9e0d85aa4322fcf64e467c41b1b3"
server_key_iv: "87ed87ce471699b6ff349fb50700e72a"
server_key_version: "encryptionkey@0"
I20260808 10:34:54.267802 6338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:54.280912 6338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:54.284984 6338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:54.285243 6338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:54.286227 6338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:54.287312 6338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:54.287369 6338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.287439 6338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:54.287469 6338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.296758 6338 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:36793
I20260808 10:34:54.296993 6582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:36793 every 8 connection(s)
I20260808 10:34:54.298079 6338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:54.301343 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 6338
I20260808 10:34:54.301578 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 5730
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.188@KRBTEST.COM
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.189@KRBTEST.COM
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:54.320760 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:41931
--local_ip_for_outbound_sockets=127.30.146.130
--tserver_master_addrs=127.30.146.190:34753
--webserver_port=42375
--webserver_interface=127.30.146.130
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab
--principal=oryx/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.146.190:34753,127.30.146.189:41421
--tserver_master_addrs=127.30.146.190:34753,127.30.146.189:41421,127.30.146.188:34901 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:34:54.321293 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.304104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:44681 (local address 127.30.146.190:34753)
0808 10:34:54.304798 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.304802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.304818 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:34:54.304867 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.304870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.304915 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.304995 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.309507 (+ 4512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.310323 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.311819 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.312232 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.312355 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.315657 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.315672 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.315675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.315727 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.317835 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.318763 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.318771 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.318776 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.318903 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.319509 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.319516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.319521 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.319841 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.319927 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.320694 (+ 767us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.320833 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595}
I20260808 10:34:54.322141 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.301453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.129:44681)
0808 10:34:54.302270 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.302287 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.303105 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.308944 (+ 5839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.308948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.309231 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.309353 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.309363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.310477 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.310483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.311580 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.311591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.312548 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.312555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.312706 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.313473 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.313494 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.315484 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.318020 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.318030 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.318053 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.318584 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.319079 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.319084 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.319086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.319364 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.320022 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.320032 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.320407 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.320652 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.321055 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":95,"threads_started":1}
I20260808 10:34:54.322825 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.303474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:56633 (local address 127.30.146.189:41421)
0808 10:34:54.303656 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.303674 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.303755 (+ 81us) server_negotiation.cc:407] Connection header received
0808 10:34:54.303872 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.303877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.303939 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.304041 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.305780 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.306904 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.308773 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.309339 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.309449 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.316955 (+ 7506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.316962 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.316968 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.317004 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.318970 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.320000 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.320006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.320011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.320101 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.320682 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.320688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.320690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.320958 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.321039 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.322447 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.322597 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:34:54.323498 6583 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34753
I20260808 10:34:54.323763 6583 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:54.324007 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.303034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:41421 (local address 127.30.146.129:56633)
0808 10:34:54.303675 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.303689 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.303780 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.304179 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.304184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.304694 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.305565 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.305589 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.307081 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.307087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.308582 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.308598 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.309651 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.309658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.309826 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.310388 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.310420 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.316776 (+ 6356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.319196 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.319201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.319206 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.319479 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.320220 (+ 741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.320226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.320231 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.320326 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.321346 (+ 1020us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.321352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.321462 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.322986 (+ 1524us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.323183 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":88,"spinlock_wait_cycles":19712,"thread_start_us":245,"threads_started":1}
I20260808 10:34:54.324378 6583 heartbeater.cc:511] Master 127.30.146.190:34753 requested a full tablet report, sending...
I20260808 10:34:54.324700 6588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.300046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:42787 (local address 127.30.146.188:34901)
0808 10:34:54.300396 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.300404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.301210 (+ 806us) server_negotiation.cc:407] Connection header received
0808 10:34:54.303215 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.303219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.303279 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.303349 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.305736 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.306730 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.308541 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.308942 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.309030 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.313896 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.313903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.313910 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.313947 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.316593 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.320452 (+ 3859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.320456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.320458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.320556 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.321614 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.321620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.321624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.321899 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.321991 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.324283 (+ 2292us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.324448 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":115,"threads_started":1}
I20260808 10:34:54.325338 6589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.300452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:34901 (local address 127.30.146.129:42787)
0808 10:34:54.300997 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.301062 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.302952 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.303792 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.303803 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.304678 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.305562 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.305584 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.306927 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.306932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.308394 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.308408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.309296 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.309300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.309521 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.310409 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.310427 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.313378 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.319614 (+ 6236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.319620 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.319624 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.319881 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.321104 (+ 1223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.321109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.321114 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.321230 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.322057 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.322063 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.322169 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.324851 (+ 2682us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.325028 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":138,"thread_start_us":123,"threads_started":1}
I20260808 10:34:54.325562 6585 heartbeater.cc:347] Connected to a master server at 127.30.146.189:41421
I20260808 10:34:54.325645 6585 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:54.325899 6585 heartbeater.cc:511] Master 127.30.146.189:41421 requested a full tablet report, sending...
I20260808 10:34:54.326294 5517 ts_manager.cc:194] Re-registered known tserver with Master: 83bc0009eb08456b9e74f477c73c7631 (127.30.146.129:36793)
I20260808 10:34:54.327050 5517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:44681
I20260808 10:34:54.327242 6171 ts_manager.cc:194] Registered new tserver with Master: 83bc0009eb08456b9e74f477c73c7631 (127.30.146.129:36793)
I20260808 10:34:54.329286 6584 heartbeater.cc:347] Connected to a master server at 127.30.146.188:34901
I20260808 10:34:54.329387 6584 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:54.329654 6584 heartbeater.cc:511] Master 127.30.146.188:34901 requested a full tablet report, sending...
I20260808 10:34:54.330988 6383 ts_manager.cc:194] Registered new tserver with Master: 83bc0009eb08456b9e74f477c73c7631 (127.30.146.129:36793)
I20260808 10:34:54.347385 6017 heartbeater.cc:503] Master 127.30.146.190:34753 was elected leader, sending a full tablet report...
W20260808 10:34:54.449702 6592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:54.449976 6592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:54.450023 6592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:54.454437 6592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:54.454521 6592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:54.454587 6592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:54.460439 6592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab
--principal=oryx/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:41931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=42375
--tserver_master_addrs=127.30.146.190:34753,127.30.146.189:41421,127.30.146.188:34901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:54.461741 6592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:54.462800 6592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:54.473555 6600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:54.473564 6603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:54.473649 6599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:54.474658 6592 server_base.cc:1034] running on GCE node
I20260808 10:34:54.475195 6592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:54.475925 6592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:54.477211 6592 hybrid_clock.cc:648] HybridClock initialized: now 1786185294477194 us; error 32 us; skew 500 ppm
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:54.485287 6592 init.cc:377] Logged in from keytab as oryx/127.30.146.130@KRBTEST.COM (short username oryx)
I20260808 10:34:54.486620 6592 webserver.cc:460] Webserver started at http://127.30.146.130:42375/ using document root <none> and password file <none>
I20260808 10:34:54.487291 6592 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:54.487402 6592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:54.491856 6592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:34:54.494300 6610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.495891 6592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:34:54.496062 6592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "285a49e71818492aa9432d21c981af40"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "dcd2bedea8cb580bd214a5e486438d52"
server_key_iv: "001eed1c324a940fcf9e731c1f1b31e5"
server_key_version: "encryptionkey@0"
I20260808 10:34:54.496948 6592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:54.518370 6592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:54.521809 6588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.517750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.30.146.188:34901)
0808 10:34:54.517891 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.517896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.518068 (+ 172us) server_negotiation.cc:407] Connection header received
0808 10:34:54.518251 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.518254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.518321 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.518425 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:54.518715 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.519409 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.520686 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.521215 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.521357 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.521514 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.521625 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.521698 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:34:54.521845 6341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.517741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:34901 (local address 127.0.0.1:41120)
0808 10:34:54.517944 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.517958 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.518125 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.518437 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.518440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.518458 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:54.518602 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.518612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.519535 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.519540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.520566 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.520581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.521358 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.521363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.521487 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.521518 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.521615 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.521703 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260808 10:34:54.522771 6394 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 10:34:54.523227 6592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:54.523424 6592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:54.523357 6428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 602f69cad51b44c6ac6edf88980a20d7. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.523511 6428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: This master's current role is: LEARNER
I20260808 10:34:54.524359 6592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:54.525475 6592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:54.525527 6592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.525569 6592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:54.525595 6592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.539883 6592 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:41931
I20260808 10:34:54.539901 6737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:41931 every 8 connection(s)
I20260808 10:34:54.541019 6592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:54.550657 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 6592
I20260808 10:34:54.550848 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 5875
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.189@KRBTEST.COM
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.188@KRBTEST.COM
I20260808 10:34:54.562199 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.543074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:44465 (local address 127.30.146.189:41421)
0808 10:34:54.543202 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.543206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.544302 (+ 1096us) server_negotiation.cc:407] Connection header received
0808 10:34:54.545656 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.545659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.545724 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.545788 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.548347 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.549118 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.550833 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.551183 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.551262 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.556159 (+ 4897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.556163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.556165 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.556196 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.558765 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.559680 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.559683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.559685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.559803 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.560280 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.560283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.560285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.560609 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.560691 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.561861 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.562019 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:34:54.562199 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.543023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:53661 (local address 127.30.146.190:34753)
0808 10:34:54.543177 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.543182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.544254 (+ 1072us) server_negotiation.cc:407] Connection header received
0808 10:34:54.545689 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.545693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.545792 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.545886 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.548049 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.548856 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.550833 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.551183 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.551262 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.556158 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.556162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.556165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.556196 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.558765 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.559681 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.559684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.559686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.559803 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.560284 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.560287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.560288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.560894 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.561013 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.561882 (+ 869us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.562028 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 10:34:54.564375 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.543425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:41421 (local address 127.30.146.130:44465)
0808 10:34:54.544197 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.544212 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.545367 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.546152 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.546162 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.546928 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.548199 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.548209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.549295 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.549298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.550687 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.550698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.551936 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.551939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.552059 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.552932 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.552944 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.555932 (+ 2988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.558923 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.558930 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.558952 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.559543 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.559921 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.559924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.559926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.560163 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.561096 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.561100 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.561504 (+ 404us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.562566 (+ 1062us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.563187 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":922,"spinlock_wait_cycles":86528,"thread_start_us":233,"threads_started":1}
I20260808 10:34:54.564388 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.543425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.130:53661)
0808 10:34:54.544092 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.544147 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.545387 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.546200 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.546202 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.547008 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.547776 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.547796 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.549002 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.549005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.550688 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.550698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.551470 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.551476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.551866 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.552784 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.552809 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.555932 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.558923 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.558930 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.558952 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.559543 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.559920 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.559924 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.559926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.560163 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.561033 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.561044 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.561504 (+ 460us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.562566 (+ 1062us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.563102 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":233,"spinlock_wait_cycles":47360,"thread_start_us":142,"threads_started":1}
I20260808 10:34:54.565233 6588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.544805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:43621 (local address 127.30.146.188:34901)
0808 10:34:54.545422 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.545426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.545443 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:34:54.545604 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.545611 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.545675 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.545728 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.548208 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.548841 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.550487 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.550855 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.550956 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.560108 (+ 9152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.560115 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.560122 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.560160 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.562665 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.563491 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.563497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.563502 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.563621 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.564045 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.564050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.564055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.564354 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.564444 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.564834 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.564980 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522}
I20260808 10:34:54.565866 6740 heartbeater.cc:347] Connected to a master server at 127.30.146.189:41421
I20260808 10:34:54.565936 6738 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34753
I20260808 10:34:54.566097 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.544455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:34901 (local address 127.30.146.130:43621)
0808 10:34:54.545120 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.545136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.545482 (+ 346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.546152 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.546162 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.547018 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.548023 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.548034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.548997 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.549000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.550314 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.550323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.554285 (+ 3962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.554295 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.554518 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.555068 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.555092 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.559899 (+ 4807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.563045 (+ 3146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.563052 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.563057 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.563359 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.563747 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.563753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.563758 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.563932 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.564508 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.564514 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.564640 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.565436 (+ 796us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.565660 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":469,"spinlock_wait_cycles":11008,"thread_start_us":119,"threads_started":1}
I20260808 10:34:54.566213 6738 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:54.566213 6740 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:54.566746 6739 heartbeater.cc:347] Connected to a master server at 127.30.146.188:34901
I20260808 10:34:54.566828 6739 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:54.566900 6740 heartbeater.cc:511] Master 127.30.146.189:41421 requested a full tablet report, sending...
I20260808 10:34:54.567024 6739 heartbeater.cc:511] Master 127.30.146.188:34901 requested a full tablet report, sending...
I20260808 10:34:54.567163 6738 heartbeater.cc:511] Master 127.30.146.190:34753 requested a full tablet report, sending...
I20260808 10:34:54.567580 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:36185
--local_ip_for_outbound_sockets=127.30.146.131
--tserver_master_addrs=127.30.146.190:34753
--webserver_port=44271
--webserver_interface=127.30.146.131
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab
--principal=oryx/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.146.190:34753,127.30.146.189:41421
--tserver_master_addrs=127.30.146.190:34753,127.30.146.189:41421,127.30.146.188:34901 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:34:54.568141 6383 ts_manager.cc:194] Registered new tserver with Master: 285a49e71818492aa9432d21c981af40 (127.30.146.130:41931)
I20260808 10:34:54.568141 6171 ts_manager.cc:194] Registered new tserver with Master: 285a49e71818492aa9432d21c981af40 (127.30.146.130:41931)
I20260808 10:34:54.568707 5517 ts_manager.cc:194] Re-registered known tserver with Master: 285a49e71818492aa9432d21c981af40 (127.30.146.130:41931)
I20260808 10:34:54.569415 5517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:53661
W20260808 10:34:54.699514 6747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:54.699800 6747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:54.699846 6747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:54.704522 6747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:54.704595 6747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:54.704664 6747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:54.710433 6747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab
--principal=oryx/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:36185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=44271
--tserver_master_addrs=127.30.146.190:34753,127.30.146.189:41421,127.30.146.188:34901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:54.711539 6747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:54.712599 6747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:54.723654 6752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:54.723634 6756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:54.723820 6754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:54.724004 6747 server_base.cc:1034] running on GCE node
I20260808 10:34:54.724478 6747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:54.725404 6747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:54.726657 6747 hybrid_clock.cc:648] HybridClock initialized: now 1786185294726620 us; error 58 us; skew 500 ppm
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:54.729825 6747 init.cc:377] Logged in from keytab as oryx/127.30.146.131@KRBTEST.COM (short username oryx)
I20260808 10:34:54.730800 6747 webserver.cc:460] Webserver started at http://127.30.146.131:44271/ using document root <none> and password file <none>
I20260808 10:34:54.731428 6747 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:54.731546 6747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:54.736495 6747 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:34:54.738914 6764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.740927 6747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:34:54.741060 6747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "729af0f3753842848716e84689418f4c"
format_stamp: "Formatted at 2026-08-08 10:34:53 on dist-test-slave-9gcw"
server_key: "431426f1bb5cbe25a7a925503dc8c1ec"
server_key_iv: "9d3f109dc21841bf52405b54f435790e"
server_key_version: "encryptionkey@0"
I20260808 10:34:54.741933 6747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:54.769488 6747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:54.774959 6747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:54.775374 6747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:54.776463 6747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:34:54.778044 6747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:54.778115 6747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.778196 6747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:54.778234 6747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:54.794140 6747 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:36185
I20260808 10:34:54.794206 6890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:36185 every 8 connection(s)
I20260808 10:34:54.795346 6747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:54.796721 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 6747
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.189@KRBTEST.COM
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:54.814574 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.799622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41514 (local address 127.30.146.190:34753)
0808 10:34:54.799949 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.799954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.800728 (+ 774us) server_negotiation.cc:407] Connection header received
0808 10:34:54.801031 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.801035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.801114 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.801289 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.802719 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.803708 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.805088 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.805511 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.805596 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.807698 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.807702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.807706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.807736 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.812042 (+ 4306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.812747 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.812754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.812758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.812869 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.813311 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.813315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.813317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.813584 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.813724 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.814177 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.814336 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"spinlock_wait_cycles":5888,"thread_start_us":81,"threads_started":1}
Aug 08 10:34:54 dist-test-slave-9gcw krb5kdc[5464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185294, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.188@KRBTEST.COM
I20260808 10:34:54.818640 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.799313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:34389 (local address 127.30.146.190:34753)
0808 10:34:54.799458 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.799464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.801238 (+ 1774us) server_negotiation.cc:407] Connection header received
0808 10:34:54.801838 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.801841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.801888 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.801940 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.804047 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.804793 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.806907 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.807237 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.807315 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.811952 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.811968 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.811971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.812025 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.813995 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.816527 (+ 2532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.816533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.816536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.816633 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.817183 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.817187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.817188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.817410 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.817498 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.818305 (+ 807us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.818445 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65,"spinlock_wait_cycles":14336}
I20260808 10:34:54.820568 6908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.799363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.131:34389)
0808 10:34:54.801038 (+ 1675us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.801049 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.801589 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.802388 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.802400 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.803101 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.803887 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.803907 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.804990 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.804995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.806653 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.806778 (+ 125us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.807515 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.807520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.807781 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.808656 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.808680 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.811771 (+ 3091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.815892 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.815902 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.815924 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.816383 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.816772 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.816777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.816780 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.817023 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.817585 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.817593 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.817974 (+ 381us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.818894 (+ 920us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.819337 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1618,"mutex_wait_us":186,"spinlock_wait_cycles":8960,"thread_start_us":130,"threads_started":1}
I20260808 10:34:54.820956 6588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.800146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:57797 (local address 127.30.146.188:34901)
0808 10:34:54.800472 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.800477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.809833 (+ 9356us) server_negotiation.cc:407] Connection header received
0808 10:34:54.809937 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.809940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.809988 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.810052 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.810635 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.811381 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.813489 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.813929 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.814021 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.817253 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.817259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.817262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.817298 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.818972 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.819505 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.819509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.819511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.819614 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.819942 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.819945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.819947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.820173 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.820259 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.820643 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.820793 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256}
I20260808 10:34:54.822021 6904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.799781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:34901 (local address 127.30.146.131:57797)
0808 10:34:54.809729 (+ 9948us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.809740 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.809860 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.810152 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.810156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.810360 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.810486 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.810495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.812355 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.812358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.813360 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.813369 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.814445 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.814449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.814557 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.815193 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.815210 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.817046 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.819085 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.819088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.819091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.819378 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.819736 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.819739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.819741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.819834 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.820324 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.820327 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.820416 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.821540 (+ 1124us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.821759 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9885,"thread_start_us":74,"threads_started":1}
I20260808 10:34:54.822263 6891 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34753
I20260808 10:34:54.818672 5517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:34:54.822532 6891 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:54.823194 6891 heartbeater.cc:511] Master 127.30.146.190:34753 requested a full tablet report, sending...
I20260808 10:34:54.823834 6910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.800266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.30.146.188:34901)
0808 10:34:54.800472 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.800477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.801515 (+ 1038us) server_negotiation.cc:407] Connection header received
0808 10:34:54.801656 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.801659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.801715 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.801803 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.803093 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.804112 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.808996 (+ 4884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.809364 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.809487 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.814580 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.814584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.814587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.814623 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.819629 (+ 5006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.821560 (+ 1931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.821563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.821565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.821650 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.821952 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.821956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.821958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.822212 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.822350 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.823542 (+ 1192us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.823672 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260808 10:34:54.824532 5516 ts_manager.cc:194] Re-registered known tserver with Master: 729af0f3753842848716e84689418f4c (127.30.146.131:36185)
I20260808 10:34:54.824649 6892 heartbeater.cc:347] Connected to a master server at 127.30.146.188:34901
I20260808 10:34:54.824746 6892 heartbeater.cc:464] Registering TS with master...
W20260808 10:34:54.824734 5517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:54.824921 6892 heartbeater.cc:511] Master 127.30.146.188:34901 requested a full tablet report, sending...
I20260808 10:34:54.825477 5516 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:34389
I20260808 10:34:54.825639 6383 ts_manager.cc:194] Registered new tserver with Master: 729af0f3753842848716e84689418f4c (127.30.146.131:36185)
I20260808 10:34:54.825673 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.800739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.30.146.189:41421)
0808 10:34:54.804335 (+ 3596us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.804340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.804351 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:34:54.804384 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.804387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.804425 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.804502 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.806525 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.807325 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.810660 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.811008 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.811106 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.813364 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.813368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.813370 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.813401 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.820948 (+ 7547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.822916 (+ 1968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.822919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.822921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.823088 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.824176 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.824180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.824182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.824409 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.824543 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.825373 (+ 830us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.825511 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3524,"thread_start_us":77,"threads_started":1}
I20260808 10:34:54.827098 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.799327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:51301 (local address 127.30.146.189:41421)
0808 10:34:54.799500 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.799504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.799945 (+ 441us) server_negotiation.cc:407] Connection header received
0808 10:34:54.801857 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.801861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.801928 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.801997 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:54.804059 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.804926 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.806825 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.807289 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.807377 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.810894 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:54.810900 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:54.810903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:54.810932 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:54.814860 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.817038 (+ 2178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.817042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.817044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.817194 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:54.817568 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:54.817571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:54.817574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:54.817856 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:54.817958 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.826821 (+ 8863us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.826946 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:34:54.827600 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.798098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:41421 (local address 127.30.146.131:51301)
0808 10:34:54.799538 (+ 1440us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.799828 (+ 290us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.801582 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.802388 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.802398 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.803108 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:54.803886 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.803907 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.805088 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.805091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.806664 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.806679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.807550 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.807557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.807778 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.808573 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:54.808598 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:54.810675 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:54.816539 (+ 5864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.816542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.816544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.816901 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.817321 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:54.817324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:54.817326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:54.817428 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:54.818030 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:54.818037 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:54.818167 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.827249 (+ 9082us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.827418 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":196,"spinlock_wait_cycles":7808,"thread_start_us":146,"threads_started":1}
I20260808 10:34:54.828029 6914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: NON_VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: true } } }
I20260808 10:34:54.828188 6914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:54.829679 6893 heartbeater.cc:347] Connected to a master server at 127.30.146.189:41421
I20260808 10:34:54.829767 6893 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:54.829945 6893 heartbeater.cc:511] Master 127.30.146.189:41421 requested a full tablet report, sending...
I20260808 10:34:54.830737 6171 ts_manager.cc:194] Registered new tserver with Master: 729af0f3753842848716e84689418f4c (127.30.146.131:36185)
I20260808 10:34:54.831502 6915 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7: attempting to promote NON_VOTER 9089e96260c7419b8e9c439d15ecb67a to VOTER
I20260808 10:34:54.831928 6915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } }
I20260808 10:34:54.833067 6187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 FOLLOWER]: Refusing update from remote peer 602f69cad51b44c6ac6edf88980a20d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 10:34:54.833119 6394 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 LEARNER]: Refusing update from remote peer 602f69cad51b44c6ac6edf88980a20d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 10:34:54.833504 6128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 10:34:54.833621 6915 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 10:34:54.835222 6915 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9089e96260c7419b8e9c439d15ecb67a (127.30.146.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:54.835908 6394 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9089e96260c7419b8e9c439d15ecb67a (127.30.146.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:54.837102 6915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:54.837249 6915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:54.837391 6915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:54.837494 6915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:54.838042 6428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:54.838153 6428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:34:54.838539 6915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:54.838678 6915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:54.839030 6187 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9089e96260c7419b8e9c439d15ecb67a (127.30.146.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:54.841337 6924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "602f69cad51b44c6ac6edf88980a20d7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:54.841498 6924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:34:54.848639 6341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.842099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41931 (local address 127.0.0.1:52314)
0808 10:34:54.842277 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.842294 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.842383 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.844399 (+ 2016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.844401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.844418 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:54.844508 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.844512 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.845983 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.845986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.847197 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.847208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.848216 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.848220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.848308 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.848331 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.848418 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.848498 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":24}
I20260808 10:34:54.848692 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.842577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52314 (local address 127.30.146.130:41931)
0808 10:34:54.843976 (+ 1399us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.843986 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.844005 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:34:54.844070 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.844075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.844264 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.844395 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:54.844641 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.845886 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.847339 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.848033 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.848195 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.848340 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.848449 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.848543 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1307,"thread_start_us":138,"threads_started":1}
I20260808 10:34:54.850387 6928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.842359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:36185 (local address 127.0.0.1:60840)
0808 10:34:54.843383 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.843398 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.843500 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.845952 (+ 2452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.845958 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.845981 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:54.846092 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.846096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.847388 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.847394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.848901 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.848912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.849971 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.849974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.850082 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.850106 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.850173 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.850247 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"mutex_wait_us":67,"thread_start_us":118,"threads_started":1}
I20260808 10:34:54.850736 6929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.843112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60840 (local address 127.30.146.131:36185)
0808 10:34:54.844760 (+ 1648us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.844778 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.844802 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:34:54.844875 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.844881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.845097 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.845278 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:54.846209 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.847303 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.849010 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.849792 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.849981 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.850189 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.850238 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.850318 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1501,"thread_start_us":143,"threads_started":1}
I20260808 10:34:54.851107 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.843033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36793 (local address 127.0.0.1:50230)
0808 10:34:54.843653 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:54.843667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:54.843763 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:54.846259 (+ 2496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:54.846262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:54.846280 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:54.846360 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.846364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.847398 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.847402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.849628 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:54.849638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.850749 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.850754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:54.850840 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.850863 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:54.850910 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 10:34:54.850975 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":109,"thread_start_us":104,"threads_started":1}
I20260808 10:34:54.851712 6930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:54.843435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50230 (local address 127.30.146.129:36793)
0808 10:34:54.845564 (+ 2129us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:54.845570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:54.845591 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:34:54.845657 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:54.845661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:54.845818 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:54.846484 (+ 666us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:54.846593 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.847245 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.849791 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:54.850593 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:54.851312 (+ 719us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:54.851436 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:54.851487 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 10:34:54.851574 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2015,"thread_start_us":76,"threads_started":1}
I20260808 10:34:54.852130 6665 tablet_service.cc:1467] Processing CreateTablet for tablet 17f29fc3dabf484bbea0f782909ebced (DEFAULT_TABLE table=test-table [id=f48635fde53e4d41a57bd3dfb43373d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:54.853294 6665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17f29fc3dabf484bbea0f782909ebced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:54.854490 6511 tablet_service.cc:1467] Processing CreateTablet for tablet 17f29fc3dabf484bbea0f782909ebced (DEFAULT_TABLE table=test-table [id=f48635fde53e4d41a57bd3dfb43373d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:54.855417 6511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17f29fc3dabf484bbea0f782909ebced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:54.860200 6821 tablet_service.cc:1467] Processing CreateTablet for tablet 17f29fc3dabf484bbea0f782909ebced (DEFAULT_TABLE table=test-table [id=f48635fde53e4d41a57bd3dfb43373d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:54.860874 6931 tablet_bootstrap.cc:492] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: Bootstrap starting.
I20260808 10:34:54.861333 6821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17f29fc3dabf484bbea0f782909ebced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:54.863227 6933 tablet_bootstrap.cc:492] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: Bootstrap starting.
I20260808 10:34:54.863845 6931 tablet_bootstrap.cc:654] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:54.864966 6931 log.cc:824] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:54.866076 6933 tablet_bootstrap.cc:654] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:54.866976 6933 log.cc:824] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:54.867496 6931 tablet_bootstrap.cc:492] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: No bootstrap required, opened a new log
I20260808 10:34:54.867700 6931 ts_tablet_manager.cc:1397] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 10:34:54.868678 6933 tablet_bootstrap.cc:492] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: No bootstrap required, opened a new log
I20260808 10:34:54.868875 6933 ts_tablet_manager.cc:1397] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:34:54.870760 6936 tablet_bootstrap.cc:492] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: Bootstrap starting.
I20260808 10:34:54.872941 6933 raft_consensus.cc:348] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:54.873286 6933 raft_consensus.cc:374] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:54.873358 6933 raft_consensus.cc:729] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83bc0009eb08456b9e74f477c73c7631, State: Initialized, Role: FOLLOWER
I20260808 10:34:54.873373 6936 tablet_bootstrap.cc:654] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:54.873327 6931 raft_consensus.cc:348] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:54.873610 6931 raft_consensus.cc:374] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:54.873670 6931 raft_consensus.cc:729] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285a49e71818492aa9432d21c981af40, State: Initialized, Role: FOLLOWER
I20260808 10:34:54.873878 6933 consensus_queue.cc:260] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:54.874121 6931 consensus_queue.cc:260] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:54.874225 6936 log.cc:824] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:54.874598 6583 heartbeater.cc:503] Master 127.30.146.190:34753 was elected leader, sending a full tablet report...
I20260808 10:34:54.874936 6933 ts_tablet_manager.cc:1428] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260808 10:34:54.875177 6931 ts_tablet_manager.cc:1428] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 10:34:54.875535 6738 heartbeater.cc:503] Master 127.30.146.190:34753 was elected leader, sending a full tablet report...
I20260808 10:34:54.876292 6936 tablet_bootstrap.cc:492] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: No bootstrap required, opened a new log
I20260808 10:34:54.876467 6936 ts_tablet_manager.cc:1397] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:34:54.881050 6936 raft_consensus.cc:348] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:54.881371 6936 raft_consensus.cc:374] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:54.881461 6936 raft_consensus.cc:729] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 729af0f3753842848716e84689418f4c, State: Initialized, Role: FOLLOWER
I20260808 10:34:54.881963 6936 consensus_queue.cc:260] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:54.882864 6891 heartbeater.cc:503] Master 127.30.146.190:34753 was elected leader, sending a full tablet report...
I20260808 10:34:54.883013 6936 ts_tablet_manager.cc:1428] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260808 10:34:55.042768 6741 tablet.cc:2369] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:34:55.047778 6894 tablet.cc:2369] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:34:55.053833 6587 tablet.cc:2369] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:34:55.084942 6938 raft_consensus.cc:482] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:55.085286 6938 raft_consensus.cc:504] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:55.086632 6938 leader_election.cc:288] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 729af0f3753842848716e84689418f4c (127.30.146.131:36185), 83bc0009eb08456b9e74f477c73c7631 (127.30.146.129:36793)
I20260808 10:34:55.091042 6929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:55.087151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:39951 (local address 127.30.146.131:36185)
0808 10:34:55.087369 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:55.087377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:55.087451 (+ 74us) server_negotiation.cc:407] Connection header received
0808 10:34:55.087623 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:55.087626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:55.087679 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:55.087811 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:55.088140 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.088723 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.089957 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.090406 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.090499 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:55.090541 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:55.090816 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 10:34:55.090898 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 10:34:55.091045 6941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:55.087045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:36185 (local address 127.30.146.130:39951)
0808 10:34:55.087344 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:55.087366 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:55.087508 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:55.087835 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:55.087841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:55.087874 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:55.088005 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:55.088016 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.088873 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.088876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:55.089823 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:55.089833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.090531 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.090535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:55.090650 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:55.090688 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:34:55.090776 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:34:55.090854 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":86,"threads_started":1}
I20260808 10:34:55.091223 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:55.087482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36793 (local address 127.30.146.130:42981)
0808 10:34:55.087723 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:55.087736 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:55.087809 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:55.088108 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:55.088111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:55.088129 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:55.088262 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:55.088268 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.089198 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.089201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:55.089981 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:55.089991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.090810 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.090813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:55.090913 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:55.090936 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:55.091013 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:34:55.091090 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":100,"threads_started":1}
I20260808 10:34:55.091324 6930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:55.087613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:42981 (local address 127.30.146.129:36793)
0808 10:34:55.087788 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:55.087794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:55.087812 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:34:55.087922 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:55.087926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:55.087976 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:55.088074 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:55.088379 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.089027 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.090123 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.090650 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.090795 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:55.090848 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:55.091061 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 10:34:55.091155 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 10:34:55.092092 6533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f29fc3dabf484bbea0f782909ebced" candidate_uuid: "285a49e71818492aa9432d21c981af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83bc0009eb08456b9e74f477c73c7631" is_pre_election: true
I20260808 10:34:55.092157 6841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f29fc3dabf484bbea0f782909ebced" candidate_uuid: "285a49e71818492aa9432d21c981af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "729af0f3753842848716e84689418f4c" is_pre_election: true
I20260808 10:34:55.092482 6533 raft_consensus.cc:2454] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285a49e71818492aa9432d21c981af40 in term 0.
I20260808 10:34:55.092517 6841 raft_consensus.cc:2454] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285a49e71818492aa9432d21c981af40 in term 0.
I20260808 10:34:55.093045 6613 leader_election.cc:302] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 285a49e71818492aa9432d21c981af40, 83bc0009eb08456b9e74f477c73c7631; no voters:
I20260808 10:34:55.093492 6938 raft_consensus.cc:2781] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:55.093618 6938 raft_consensus.cc:482] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:55.093679 6938 raft_consensus.cc:3037] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:55.094748 6938 raft_consensus.cc:504] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:55.095108 6938 leader_election.cc:288] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [CANDIDATE]: Term 1 election: Requested vote from peers 729af0f3753842848716e84689418f4c (127.30.146.131:36185), 83bc0009eb08456b9e74f477c73c7631 (127.30.146.129:36793)
I20260808 10:34:55.095570 6841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f29fc3dabf484bbea0f782909ebced" candidate_uuid: "285a49e71818492aa9432d21c981af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "729af0f3753842848716e84689418f4c"
I20260808 10:34:55.095769 6841 raft_consensus.cc:3037] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:55.095805 6533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f29fc3dabf484bbea0f782909ebced" candidate_uuid: "285a49e71818492aa9432d21c981af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83bc0009eb08456b9e74f477c73c7631"
I20260808 10:34:55.095953 6533 raft_consensus.cc:3037] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:55.097278 6841 raft_consensus.cc:2454] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285a49e71818492aa9432d21c981af40 in term 1.
I20260808 10:34:55.097429 6533 raft_consensus.cc:2454] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285a49e71818492aa9432d21c981af40 in term 1.
I20260808 10:34:55.097725 6613 leader_election.cc:302] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 285a49e71818492aa9432d21c981af40, 729af0f3753842848716e84689418f4c; no voters:
I20260808 10:34:55.097954 6938 raft_consensus.cc:2781] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:55.098214 6938 raft_consensus.cc:686] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 1 LEADER]: Becoming Leader. State: Replica: 285a49e71818492aa9432d21c981af40, State: Running, Role: LEADER
I20260808 10:34:55.098609 6938 consensus_queue.cc:237] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:55.102731 5516 catalog_manager.cc:5495] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 285a49e71818492aa9432d21c981af40 (127.30.146.130). New cstate: current_term: 1 leader_uuid: "285a49e71818492aa9432d21c981af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:55.143009 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:55.139247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52320 (local address 127.30.146.130:41931)
0808 10:34:55.139409 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:55.139414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:55.139442 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:34:55.139540 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:55.139542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:55.139598 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:55.139716 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:55.140001 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.140509 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.141476 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.141696 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.141817 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:55.142144 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:55.142569 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:55.142701 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:55.142786 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 10:34:55.142848 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:34:55.149961 6841 raft_consensus.cc:1261] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c [term 1 FOLLOWER]: Refusing update from remote peer 285a49e71818492aa9432d21c981af40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:55.150053 6533 raft_consensus.cc:1261] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631 [term 1 FOLLOWER]: Refusing update from remote peer 285a49e71818492aa9432d21c981af40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:55.150758 6938 consensus_queue.cc:1035] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:55.151034 6944 consensus_queue.cc:1035] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:55.161744 6950 mvcc.cc:205] Tried to move back new op lower bound from 7316214968924758016 to 7316214968727035904. Current Snapshot: MvccSnapshot[applied={T|T < 7316214968924758016}]
I20260808 10:34:55.170655 6949 mvcc.cc:205] Tried to move back new op lower bound from 7316214968924758016 to 7316214968727035904. Current Snapshot: MvccSnapshot[applied={T|T < 7316214968924758016}]
I20260808 10:34:55.173450 5517 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:55.173616 5517 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:55.179183 5517 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 602f69cad51b44c6ac6edf88980a20d7: Sending DeleteTablet for 3 replicas of tablet 17f29fc3dabf484bbea0f782909ebced
I20260808 10:34:55.179903 6665 tablet_service.cc:1514] Processing DeleteTablet for tablet 17f29fc3dabf484bbea0f782909ebced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:55 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:52314
I20260808 10:34:55.180172 6511 tablet_service.cc:1514] Processing DeleteTablet for tablet 17f29fc3dabf484bbea0f782909ebced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:55 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:50230
I20260808 10:34:55.180877 6821 tablet_service.cc:1514] Processing DeleteTablet for tablet 17f29fc3dabf484bbea0f782909ebced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:55 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:60840
I20260808 10:34:55.183245 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 5485
I20260808 10:34:55.189429 6961 tablet_replica.cc:330] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: stopping tablet replica
I20260808 10:34:55.189836 6961 raft_consensus.cc:2229] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:55.190192 6961 raft_consensus.cc:2258] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:55.190619 6959 tablet_replica.cc:330] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: stopping tablet replica
I20260808 10:34:55.190948 6959 raft_consensus.cc:2229] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:55.191260 6959 raft_consensus.cc:2258] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:55.191634 6958 tablet_replica.cc:330] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: stopping tablet replica
I20260808 10:34:55.191927 6958 raft_consensus.cc:2229] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:34:55.192288 6958 raft_consensus.cc:2258] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:55.192981 6961 ts_tablet_manager.cc:1905] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:55.193884 6958 ts_tablet_manager.cc:1905] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:55.196780 6958 ts_tablet_manager.cc:1918] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:55.196879 6958 log.cc:1196] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/17f29fc3dabf484bbea0f782909ebced
I20260808 10:34:55.197229 6958 ts_tablet_manager.cc:1939] T 17f29fc3dabf484bbea0f782909ebced P 285a49e71818492aa9432d21c981af40: Deleting consensus metadata
W20260808 10:34:55.198637 6613 connection.cc:535] server connection from 127.0.0.1:52314 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:34:55.199898 6959 ts_tablet_manager.cc:1905] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:55.202577 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:55.181759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47740 (local address 127.30.146.189:41421)
0808 10:34:55.181858 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:55.181862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:55.181875 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:34:55.181912 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:55.181915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:55.181957 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:55.182039 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:55.183145 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.183911 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.187191 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.187586 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.187681 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:55.196592 (+ 8911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:55.196597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:55.196604 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:55.196640 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:55.198887 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:55.200199 (+ 1312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:55.200203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:55.200205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:55.200308 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:55.201428 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:55.201435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:55.201440 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:55.201745 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:55.201886 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:55.202286 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 10:34:55.202421 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:34:55.202863 6959 ts_tablet_manager.cc:1918] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:55.202972 6959 log.cc:1196] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/17f29fc3dabf484bbea0f782909ebced
I20260808 10:34:55.203325 6959 ts_tablet_manager.cc:1939] T 17f29fc3dabf484bbea0f782909ebced P 83bc0009eb08456b9e74f477c73c7631: Deleting consensus metadata
I20260808 10:34:55.204294 6961 ts_tablet_manager.cc:1918] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:55.204416 6961 log.cc:1196] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/17f29fc3dabf484bbea0f782909ebced
W20260808 10:34:55.204481 6457 connection.cc:441] server connection from 127.0.0.1:50230 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50230 (request call id 1) could send its response
I20260808 10:34:55.205210 6961 ts_tablet_manager.cc:1939] T 17f29fc3dabf484bbea0f782909ebced P 729af0f3753842848716e84689418f4c: Deleting consensus metadata
W20260808 10:34:55.206763 6769 connection.cc:441] server connection from 127.0.0.1:60840 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60840 (request call id 1) could send its response
I20260808 10:34:55.215643 6910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:55.181531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.30.146.188:34901)
0808 10:34:55.181630 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:55.181635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:55.203564 (+ 21929us) server_negotiation.cc:407] Connection header received
0808 10:34:55.204687 (+ 1123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:55.204694 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:55.204755 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:55.204863 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:55.205860 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.206593 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.208134 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.208591 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.208689 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:55.210607 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:55.210611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:55.210617 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:55.210646 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:55.212995 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:55.213506 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:55.213511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:55.213513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:55.213617 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:55.213972 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:55.213977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:55.213980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:55.214234 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:55.214369 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:55.215264 (+ 895us) server_negotiation.cc:299] Negotiation successful
0808 10:34:55.215438 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":36}
I20260808 10:34:55.216097 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:55.205856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.30.146.189:41421)
0808 10:34:55.205956 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:55.205961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:55.206853 (+ 892us) server_negotiation.cc:407] Connection header received
0808 10:34:55.207036 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:55.207040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:55.207093 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:55.207133 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:55.207940 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.208674 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.210088 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.210455 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.210540 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:55.212522 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:55.212525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:55.212528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:55.212555 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:55.214225 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:55.214701 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:55.214704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:55.214706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:55.214789 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:55.215079 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:55.215082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:55.215084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:55.215356 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:55.215463 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:55.215828 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 10:34:55.215949 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:34:55.216781 6588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:55.205453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41142 (local address 127.30.146.188:34901)
0808 10:34:55.205582 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:55.205588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:55.206755 (+ 1167us) server_negotiation.cc:407] Connection header received
0808 10:34:55.206869 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:55.206872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:55.206919 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:55.206992 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:55.208339 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.209196 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.210462 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:55.210801 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:55.210870 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:55.212526 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:55.212530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:55.212532 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:55.212558 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:55.214932 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:55.215434 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:55.215437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:55.215438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:55.215513 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:55.215841 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:55.215844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:55.215846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:55.216039 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:55.216139 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:55.216496 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 10:34:55.216605 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":52}
I20260808 10:34:56.166276 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.165793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.129:48901)
0808 10:34:56.166070 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.166176 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":78,"threads_started":1}
W20260808 10:34:56.166643 6583 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:34753 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
I20260808 10:34:56.169335 6971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.168800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.130:45075)
0808 10:34:56.169076 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.169212 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":70,"threads_started":1}
W20260808 10:34:56.169682 6738 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:34753 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
I20260808 10:34:56.172034 6972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.171528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.131:43189)
0808 10:34:56.171754 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.171954 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
W20260808 10:34:56.172309 6891 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:34753 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
I20260808 10:34:56.679251 6973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 602f69cad51b44c6ac6edf88980a20d7)
I20260808 10:34:56.679574 6973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } }
I20260808 10:34:56.679874 6974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 602f69cad51b44c6ac6edf88980a20d7)
I20260808 10:34:56.680116 6974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } }
I20260808 10:34:56.681015 6973 leader_election.cc:288] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 602f69cad51b44c6ac6edf88980a20d7 (127.30.146.190:34753), 36e1691b6d774580985c7028e4901761 (127.30.146.189:41421)
I20260808 10:34:56.681568 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.681085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:41998)
0808 10:34:56.681362 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.681452 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":88,"threads_started":1}
W20260808 10:34:56.682022 6369 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
I20260808 10:34:56.682096 6979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.681692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:42006)
0808 10:34:56.681929 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.682000 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":102,"threads_started":1}
I20260808 10:34:56.682440 6974 leader_election.cc:288] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 602f69cad51b44c6ac6edf88980a20d7 (127.30.146.190:34753), 9089e96260c7419b8e9c439d15ecb67a (127.30.146.188:34901)
W20260808 10:34:56.682477 6157 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
I20260808 10:34:56.683485 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.683074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:42020)
0808 10:34:56.683352 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.683404 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":88,"threads_started":1}
W20260808 10:34:56.683611 6157 leader_election.cc:334] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 602f69cad51b44c6ac6edf88980a20d7 (127.30.146.190:34753): Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
I20260808 10:34:56.684219 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.683952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:42024)
0808 10:34:56.684086 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.684137 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260808 10:34:56.684398 6369 leader_election.cc:334] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 602f69cad51b44c6ac6edf88980a20d7 (127.30.146.190:34753): Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
I20260808 10:34:56.686408 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.681350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:41421 (local address 127.0.0.1:47764)
0808 10:34:56.681650 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.681665 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:56.681789 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:56.682086 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:56.682090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:56.682107 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:56.682229 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:56.682245 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.683426 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.683430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:56.684983 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:56.684994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.686021 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.686026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:56.686119 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.686142 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:56.686222 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:34:56.686284 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":4,"thread_start_us":95,"threads_started":1}
I20260808 10:34:56.686437 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.681318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47764 (local address 127.30.146.189:41421)
0808 10:34:56.681560 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:56.681565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:56.681759 (+ 194us) server_negotiation.cc:407] Connection header received
0808 10:34:56.681917 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:56.681920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:56.681969 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:56.682076 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:56.682343 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.683269 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.685180 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.685861 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.685985 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.686037 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:56.686252 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 10:34:56.686323 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":59,"threads_started":1}
I20260808 10:34:56.687043 6187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9089e96260c7419b8e9c439d15ecb67a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "36e1691b6d774580985c7028e4901761" is_pre_election: true
I20260808 10:34:56.687274 6187 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9089e96260c7419b8e9c439d15ecb67a in term 1.
I20260808 10:34:56.687727 6979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.682917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:34901 (local address 127.0.0.1:41148)
0808 10:34:56.683178 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.683197 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:56.683296 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:56.683982 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:56.683985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:56.684005 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:56.684100 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:56.684107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.685223 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.685226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:56.686417 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:56.686426 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.687264 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.687267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:56.687390 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.687418 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 10:34:56.687529 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:34:56.687630 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260808 10:34:56.687688 6366 leader_election.cc:302] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36e1691b6d774580985c7028e4901761, 9089e96260c7419b8e9c439d15ecb67a; no voters: 602f69cad51b44c6ac6edf88980a20d7
I20260808 10:34:56.687795 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.683114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.30.146.188:34901)
0808 10:34:56.683735 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:56.683745 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:56.683759 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:34:56.683800 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:56.683803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:56.683846 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:56.683939 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:56.684226 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.685092 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.686562 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.687104 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.687226 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.687285 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:56.687548 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 10:34:56.687636 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":557,"thread_start_us":104,"threads_started":1}
I20260808 10:34:56.688094 6973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 10:34:56.688165 6973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 602f69cad51b44c6ac6edf88980a20d7)
I20260808 10:34:56.688208 6973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:34:56.688314 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36e1691b6d774580985c7028e4901761" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9089e96260c7419b8e9c439d15ecb67a" is_pre_election: true
I20260808 10:34:56.689455 6973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } }
I20260808 10:34:56.689713 6394 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36e1691b6d774580985c7028e4901761 in current term 2: Already voted for candidate 9089e96260c7419b8e9c439d15ecb67a in this term.
I20260808 10:34:56.689868 6973 leader_election.cc:288] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [CANDIDATE]: Term 2 election: Requested vote from peers 602f69cad51b44c6ac6edf88980a20d7 (127.30.146.190:34753), 36e1691b6d774580985c7028e4901761 (127.30.146.189:41421)
I20260808 10:34:56.690213 6187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9089e96260c7419b8e9c439d15ecb67a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "36e1691b6d774580985c7028e4901761"
I20260808 10:34:56.690281 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.689950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:42036)
0808 10:34:56.690187 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.690230 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260808 10:34:56.690347 6187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:34:56.690786 6157 leader_election.cc:302] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36e1691b6d774580985c7028e4901761; no voters: 602f69cad51b44c6ac6edf88980a20d7, 9089e96260c7419b8e9c439d15ecb67a
I20260808 10:34:56.690961 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.690742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:42050)
0808 10:34:56.690844 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.690899 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:34:56.691130 6369 leader_election.cc:334] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 602f69cad51b44c6ac6edf88980a20d7 (127.30.146.190:34753): Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
I20260808 10:34:56.691601 6187 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9089e96260c7419b8e9c439d15ecb67a in term 2.
I20260808 10:34:56.691746 6974 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:34:56.691912 6366 leader_election.cc:302] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36e1691b6d774580985c7028e4901761, 9089e96260c7419b8e9c439d15ecb67a; no voters: 602f69cad51b44c6ac6edf88980a20d7
I20260808 10:34:56.692112 6973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:34:56.692355 6973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [term 2 LEADER]: Becoming Leader. State: Replica: 9089e96260c7419b8e9c439d15ecb67a, State: Running, Role: LEADER
I20260808 10:34:56.692606 6973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } }
I20260808 10:34:56.693861 6984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9089e96260c7419b8e9c439d15ecb67a. Latest consensus state: current_term: 2 leader_uuid: "9089e96260c7419b8e9c439d15ecb67a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:56.693998 6984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:56.694345 6986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:56.697105 6986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:56.697556 6986 catalog_manager.cc:1259] Loaded cluster ID: 6f9ec934f36e41308f35622429feb24c
I20260808 10:34:56.697630 6986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:56.698302 6986 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:56.698690 6986 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Loaded TSK: 0
I20260808 10:34:56.699452 6986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:34:56.737627 6382 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:34:56.740688 6382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:56.743822 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.743550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:42062)
0808 10:34:56.743684 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.743747 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 10:34:56.743896 6187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [term 2 FOLLOWER]: Refusing update from remote peer 9089e96260c7419b8e9c439d15ecb67a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 10:34:56.744607 6984 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [LEADER]: Connected to new peer: Peer: permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 10:34:56.745038 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.744783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.0.0.1:42074)
0808 10:34:56.744884 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.744943 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 10:34:56.745265 6369 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a -> Peer 602f69cad51b44c6ac6edf88980a20d7 (127.30.146.190:34753): Couldn't send request to peer 602f69cad51b44c6ac6edf88980a20d7. Status: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 10:34:56.746842 6974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9089e96260c7419b8e9c439d15ecb67a. Latest consensus state: current_term: 2 leader_uuid: "9089e96260c7419b8e9c439d15ecb67a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:56.746955 6974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:34:56.748175 6984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9089e96260c7419b8e9c439d15ecb67a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:56.748311 6984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:56.748486 6974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9089e96260c7419b8e9c439d15ecb67a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "602f69cad51b44c6ac6edf88980a20d7" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34753 } } peers { permanent_uuid: "36e1691b6d774580985c7028e4901761" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 41421 } attrs { promote: false } } peers { permanent_uuid: "9089e96260c7419b8e9c439d15ecb67a" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 34901 } attrs { promote: false } } }
I20260808 10:34:56.748606 6974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e1691b6d774580985c7028e4901761 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:34:56.759692 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.755175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41931 (local address 127.0.0.1:52322)
0808 10:34:56.755338 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.755355 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:56.755441 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:56.756015 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:56.756018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:56.756040 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:56.756172 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:56.756184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.757413 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.757418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:56.758206 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:56.758239 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.759192 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.759196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:56.759296 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.759323 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:34:56.759426 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:34:56.759530 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":2}
I20260808 10:34:56.759723 6993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.755359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52322 (local address 127.30.146.130:41931)
0808 10:34:56.755715 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:56.755721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:56.755737 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:34:56.755795 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:56.755798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:56.755861 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:56.755998 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:56.756336 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.757252 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.758343 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.759079 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.759211 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.759280 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:56.759468 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 10:34:56.759566 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":98,"threads_started":1}
I20260808 10:34:56.760504 6665 tablet_service.cc:1467] Processing CreateTablet for tablet 86f8ed59d72249a7a8ee98f4dc97b9b8 (DEFAULT_TABLE table=test-table [id=571ce1b1de76498cb555c6287dd52920]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:56.760846 6665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f8ed59d72249a7a8ee98f4dc97b9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:56.760844 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.755736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:36185 (local address 127.0.0.1:60850)
0808 10:34:56.756266 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.756282 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:56.756362 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:56.756967 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:56.756970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:56.756991 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:56.757080 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:56.757085 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.758281 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.758287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:56.759524 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:56.759532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.760466 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.760470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:56.760557 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.760581 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:34:56.760646 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:34:56.760723 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":90}
I20260808 10:34:56.761312 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.755958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36793 (local address 127.0.0.1:50234)
0808 10:34:56.756385 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.756398 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:56.756482 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:56.757245 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:56.757247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:56.757264 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:56.757360 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:56.757368 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.758465 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.758469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:56.759865 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:56.759876 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.760848 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.760854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:56.760952 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.760978 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:34:56.761053 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:34:56.761119 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":31,"thread_start_us":114,"threads_started":1}
I20260808 10:34:56.761559 6994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.755950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.30.146.131:36185)
0808 10:34:56.756680 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:56.756684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:56.756695 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:34:56.756749 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:56.756751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:56.756835 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:56.756927 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:56.757251 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.758154 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.759666 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.760326 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.761123 (+ 797us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.761196 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:56.761232 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 10:34:56.761301 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":108,"threads_started":1}
I20260808 10:34:56.761726 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.756629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50234 (local address 127.30.146.129:36793)
0808 10:34:56.756960 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:56.756964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:56.756977 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:34:56.757030 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:56.757033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:56.757075 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:56.757234 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:56.757473 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.758335 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.759995 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.760607 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.760732 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.760790 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:56.761479 (+ 689us) server_negotiation.cc:299] Negotiation successful
0808 10:34:56.761569 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":110,"threads_started":1}
I20260808 10:34:56.762055 6821 tablet_service.cc:1467] Processing CreateTablet for tablet 86f8ed59d72249a7a8ee98f4dc97b9b8 (DEFAULT_TABLE table=test-table [id=571ce1b1de76498cb555c6287dd52920]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:56.762323 6821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f8ed59d72249a7a8ee98f4dc97b9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:56.762290 6511 tablet_service.cc:1467] Processing CreateTablet for tablet 86f8ed59d72249a7a8ee98f4dc97b9b8 (DEFAULT_TABLE table=test-table [id=571ce1b1de76498cb555c6287dd52920]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:56.762535 6511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f8ed59d72249a7a8ee98f4dc97b9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:56.763883 7000 tablet_bootstrap.cc:492] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40: Bootstrap starting.
I20260808 10:34:56.764855 7000 tablet_bootstrap.cc:654] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:56.765008 7001 tablet_bootstrap.cc:492] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c: Bootstrap starting.
I20260808 10:34:56.765401 7002 tablet_bootstrap.cc:492] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631: Bootstrap starting.
I20260808 10:34:56.766093 7001 tablet_bootstrap.cc:654] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:56.766353 7002 tablet_bootstrap.cc:654] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:56.766402 7000 tablet_bootstrap.cc:492] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40: No bootstrap required, opened a new log
I20260808 10:34:56.766502 7000 ts_tablet_manager.cc:1397] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:56.767048 7000 raft_consensus.cc:348] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:56.767185 7000 raft_consensus.cc:374] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:56.767230 7000 raft_consensus.cc:729] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285a49e71818492aa9432d21c981af40, State: Initialized, Role: FOLLOWER
I20260808 10:34:56.767360 7000 consensus_queue.cc:260] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:56.767408 7001 tablet_bootstrap.cc:492] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c: No bootstrap required, opened a new log
I20260808 10:34:56.767521 7001 ts_tablet_manager.cc:1397] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:56.767556 7002 tablet_bootstrap.cc:492] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631: No bootstrap required, opened a new log
I20260808 10:34:56.767652 7002 ts_tablet_manager.cc:1397] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:34:56.767792 7000 ts_tablet_manager.cc:1428] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:34:56.768025 7001 raft_consensus.cc:348] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:56.768090 7002 raft_consensus.cc:348] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:56.768164 7001 raft_consensus.cc:374] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:56.768183 7002 raft_consensus.cc:374] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:56.768204 7001 raft_consensus.cc:729] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 729af0f3753842848716e84689418f4c, State: Initialized, Role: FOLLOWER
I20260808 10:34:56.768221 7002 raft_consensus.cc:729] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83bc0009eb08456b9e74f477c73c7631, State: Initialized, Role: FOLLOWER
I20260808 10:34:56.768332 7001 consensus_queue.cc:260] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:56.768368 7002 consensus_queue.cc:260] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:56.768664 7001 ts_tablet_manager.cc:1428] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:34:56.768679 7002 ts_tablet_manager.cc:1428] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:56.771031 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.770592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.131:50467)
0808 10:34:56.770867 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.770929 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":106,"threads_started":1}
I20260808 10:34:56.771598 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.768264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.130:48637)
0808 10:34:56.771430 (+ 3166us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.771496 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3087,"thread_start_us":102,"threads_started":1}
I20260808 10:34:56.771826 6584 heartbeater.cc:503] Master 127.30.146.188:34901 was elected leader, sending a full tablet report...
I20260808 10:34:56.772410 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.772079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.131:42931)
0808 10:34:56.772289 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.772342 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 10:34:56.771652 6739 heartbeater.cc:503] Master 127.30.146.188:34901 was elected leader, sending a full tablet report...
I20260808 10:34:56.773559 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.769358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.129:46049)
0808 10:34:56.773389 (+ 4031us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.773450 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3947,"thread_start_us":92,"threads_started":1}
I20260808 10:34:56.773923 6892 heartbeater.cc:503] Master 127.30.146.188:34901 was elected leader, sending a full tablet report...
I20260808 10:34:56.774945 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.773270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.130:46787)
0808 10:34:56.774811 (+ 1541us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.774873 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1443}
I20260808 10:34:56.775308 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.774096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.129:36239)
0808 10:34:56.775152 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.775236 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":966}
I20260808 10:34:56.840749 7007 raft_consensus.cc:482] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:56.840903 7007 raft_consensus.cc:504] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:56.841375 7007 leader_election.cc:288] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 729af0f3753842848716e84689418f4c (127.30.146.131:36185), 83bc0009eb08456b9e74f477c73c7631 (127.30.146.129:36793)
I20260808 10:34:56.841974 6841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f8ed59d72249a7a8ee98f4dc97b9b8" candidate_uuid: "285a49e71818492aa9432d21c981af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "729af0f3753842848716e84689418f4c" is_pre_election: true
I20260808 10:34:56.842103 6533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f8ed59d72249a7a8ee98f4dc97b9b8" candidate_uuid: "285a49e71818492aa9432d21c981af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83bc0009eb08456b9e74f477c73c7631" is_pre_election: true
I20260808 10:34:56.842133 6841 raft_consensus.cc:2454] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285a49e71818492aa9432d21c981af40 in term 0.
I20260808 10:34:56.842312 6533 raft_consensus.cc:2454] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285a49e71818492aa9432d21c981af40 in term 0.
I20260808 10:34:56.842697 6613 leader_election.cc:302] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 285a49e71818492aa9432d21c981af40, 729af0f3753842848716e84689418f4c; no voters:
I20260808 10:34:56.842909 7007 raft_consensus.cc:2781] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:56.842975 7007 raft_consensus.cc:482] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:56.843006 7007 raft_consensus.cc:3037] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:56.844338 7007 raft_consensus.cc:504] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:56.844733 7007 leader_election.cc:288] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [CANDIDATE]: Term 1 election: Requested vote from peers 729af0f3753842848716e84689418f4c (127.30.146.131:36185), 83bc0009eb08456b9e74f477c73c7631 (127.30.146.129:36793)
I20260808 10:34:56.845273 6841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f8ed59d72249a7a8ee98f4dc97b9b8" candidate_uuid: "285a49e71818492aa9432d21c981af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "729af0f3753842848716e84689418f4c"
I20260808 10:34:56.845460 6841 raft_consensus.cc:3037] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:56.845429 6533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f8ed59d72249a7a8ee98f4dc97b9b8" candidate_uuid: "285a49e71818492aa9432d21c981af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83bc0009eb08456b9e74f477c73c7631"
I20260808 10:34:56.845557 6533 raft_consensus.cc:3037] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:56.846872 6841 raft_consensus.cc:2454] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285a49e71818492aa9432d21c981af40 in term 1.
I20260808 10:34:56.846872 6533 raft_consensus.cc:2454] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285a49e71818492aa9432d21c981af40 in term 1.
I20260808 10:34:56.847218 6613 leader_election.cc:302] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 285a49e71818492aa9432d21c981af40, 83bc0009eb08456b9e74f477c73c7631; no voters:
I20260808 10:34:56.847462 7007 raft_consensus.cc:2781] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:56.847718 7007 raft_consensus.cc:686] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 1 LEADER]: Becoming Leader. State: Replica: 285a49e71818492aa9432d21c981af40, State: Running, Role: LEADER
I20260808 10:34:56.847896 7007 consensus_queue.cc:237] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } }
I20260808 10:34:56.848589 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.848282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.130:43949)
0808 10:34:56.848427 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.848496 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 10:34:56.850219 6383 catalog_manager.cc:5495] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 285a49e71818492aa9432d21c981af40 (127.30.146.130). New cstate: current_term: 1 leader_uuid: "285a49e71818492aa9432d21c981af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285a49e71818492aa9432d21c981af40" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:56.885212 6993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.882136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52330 (local address 127.30.146.130:41931)
0808 10:34:56.882271 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:56.882275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:56.882290 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:34:56.882405 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:56.882407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:56.882454 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:56.882522 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:56.882777 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.883281 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.884139 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:56.884360 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:56.884459 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:56.884701 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:56.884774 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:56.884827 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:56.884982 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:34:56.885034 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:34:56.887874 6533 raft_consensus.cc:1261] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631 [term 1 FOLLOWER]: Refusing update from remote peer 285a49e71818492aa9432d21c981af40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:56.887868 6841 raft_consensus.cc:1261] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c [term 1 FOLLOWER]: Refusing update from remote peer 285a49e71818492aa9432d21c981af40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:56.888365 7007 consensus_queue.cc:1035] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "729af0f3753842848716e84689418f4c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:56.888562 7015 consensus_queue.cc:1035] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83bc0009eb08456b9e74f477c73c7631" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:56.890153 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.889853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.131:39195)
0808 10:34:56.889985 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.890055 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:34:56.891140 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.890828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.129:32789)
0808 10:34:56.890987 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.891048 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 10:34:56.895016 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.894166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.130:55421)
0808 10:34:56.894855 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.894934 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606}
I20260808 10:34:56.895156 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.894562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.129:38389)
0808 10:34:56.895001 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.895071 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372}
I20260808 10:34:56.895855 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.895584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.131:52379)
0808 10:34:56.895738 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.895788 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 10:34:56.898682 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.898440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34753 (local address 127.30.146.130:52143)
0808 10:34:56.898557 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.898609 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 10:34:56.900722 6382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:56.900943 6382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:56.905025 6382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9089e96260c7419b8e9c439d15ecb67a: Sending DeleteTablet for 3 replicas of tablet 86f8ed59d72249a7a8ee98f4dc97b9b8
I20260808 10:34:56.905978 6665 tablet_service.cc:1514] Processing DeleteTablet for tablet 86f8ed59d72249a7a8ee98f4dc97b9b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:56 UTC) from {username='oryx', principal='oryx/127.30.146.188@KRBTEST.COM'} at 127.0.0.1:52322
I20260808 10:34:56.906004 6511 tablet_service.cc:1514] Processing DeleteTablet for tablet 86f8ed59d72249a7a8ee98f4dc97b9b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:56 UTC) from {username='oryx', principal='oryx/127.30.146.188@KRBTEST.COM'} at 127.0.0.1:50234
I20260808 10:34:56.905984 6821 tablet_service.cc:1514] Processing DeleteTablet for tablet 86f8ed59d72249a7a8ee98f4dc97b9b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:56 UTC) from {username='oryx', principal='oryx/127.30.146.188@KRBTEST.COM'} at 127.0.0.1:60850
I20260808 10:34:56.906579 7029 tablet_replica.cc:330] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 83bc0009eb08456b9e74f477c73c7631: stopping tablet replica
I20260808 10:34:56.906803 7028 tablet_replica.cc:330] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c: stopping tablet replica
I20260808 10:34:56.906948 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 6338
I20260808 10:34:56.906981 7028 raft_consensus.cc:2229] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:56.907135 7028 raft_consensus.cc:2258] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:56.908180 7030 tablet_replica.cc:330] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40: stopping tablet replica
I20260808 10:34:56.908313 7030 raft_consensus.cc:2229] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:34:56.908528 7030 raft_consensus.cc:2258] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:56.909665 7028 ts_tablet_manager.cc:1905] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:56.910044 7030 ts_tablet_manager.cc:1905] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:56.912813 7030 ts_tablet_manager.cc:1918] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:56.912813 7028 ts_tablet_manager.cc:1918] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:56.912915 7030 log.cc:1196] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/86f8ed59d72249a7a8ee98f4dc97b9b8
I20260808 10:34:56.912915 7028 log.cc:1196] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/86f8ed59d72249a7a8ee98f4dc97b9b8
I20260808 10:34:56.913264 7028 ts_tablet_manager.cc:1939] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 729af0f3753842848716e84689418f4c: Deleting consensus metadata
I20260808 10:34:56.913282 7030 ts_tablet_manager.cc:1939] T 86f8ed59d72249a7a8ee98f4dc97b9b8 P 285a49e71818492aa9432d21c981af40: Deleting consensus metadata
I20260808 10:34:56.914144 6367 catalog_manager.cc:4841] TS 729af0f3753842848716e84689418f4c (127.30.146.131:36185): tablet 86f8ed59d72249a7a8ee98f4dc97b9b8 (table test-table [id=571ce1b1de76498cb555c6287dd52920]) successfully deleted
I20260808 10:34:56.914243 6369 catalog_manager.cc:4841] TS 285a49e71818492aa9432d21c981af40 (127.30.146.130:41931): tablet 86f8ed59d72249a7a8ee98f4dc97b9b8 (table test-table [id=571ce1b1de76498cb555c6287dd52920]) successfully deleted
I20260808 10:34:56.920485 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 6592
I20260808 10:34:56.920754 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:56.920456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36793 (local address 127.0.0.1:50250)
0808 10:34:56.920593 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:56.920655 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 10:34:56.923691 6367 catalog_manager.cc:4584] TS 83bc0009eb08456b9e74f477c73c7631 (127.30.146.129:36793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86f8ed59d72249a7a8ee98f4dc97b9b8: Network error: Client connection negotiation failed: client connection to 127.30.146.129:36793: connect: Connection refused (error 111)
I20260808 10:34:56.933506 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 6747
I20260808 10:34:56.941334 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 6023
I20260808 10:34:56.949059 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 6133
2026-08-08T10:34:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5890 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:34:57 dist-test-slave-9gcw krb5kdc[7033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:34:57 dist-test-slave-9gcw krb5kdc[7033](info): set up 2 sockets
Aug 08 10:34:57 dist-test-slave-9gcw krb5kdc[7033](info): commencing operation
krb5kdc: starting...
W20260808 10:34:58.049728 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:34:58 dist-test-slave-9gcw krb5kdc[7033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:34:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:34:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.190.keytab.
Entry for principal oryx/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.190.keytab.
I20260808 10:34:58.184612 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:35407
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.190.keytab
--principal=oryx/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:35407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:58.299823 7048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:58.300092 7048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:58.300134 7048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:58.304186 7048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:34:58.304255 7048 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:34:58.304272 7048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:58.304291 7048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:58.304307 7048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:34:58.304328 7048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:34:58.309940 7048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:35407
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.190.keytab
--principal=oryx/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:35407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:58.310974 7048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:58.311970 7048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:58.320819 7053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:58.320837 7054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:58.320940 7056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:58.321224 7048 server_base.cc:1034] running on GCE node
I20260808 10:34:58.321849 7048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:58.323086 7048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:58.324278 7048 hybrid_clock.cc:648] HybridClock initialized: now 1786185298324258 us; error 32 us; skew 500 ppm
Aug 08 10:34:58 dist-test-slave-9gcw krb5kdc[7033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185298, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:58.327360 7048 init.cc:377] Logged in from keytab as oryx/127.30.146.190@KRBTEST.COM (short username oryx)
I20260808 10:34:58.328307 7048 webserver.cc:460] Webserver started at http://127.30.146.190:46635/ using document root <none> and password file <none>
I20260808 10:34:58.328887 7048 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:58.328953 7048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:58.329276 7048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:58.331408 7048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "6faa9f501ce5474cbdec7d05706e88eb"
format_stamp: "Formatted at 2026-08-08 10:34:58 on dist-test-slave-9gcw"
server_key: "1249a530f3e0f4b3b6e9be843fea401f"
server_key_iv: "63ea44c52d30963588b10fa3110c685b"
server_key_version: "encryptionkey@0"
I20260808 10:34:58.331987 7048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "6faa9f501ce5474cbdec7d05706e88eb"
format_stamp: "Formatted at 2026-08-08 10:34:58 on dist-test-slave-9gcw"
server_key: "1249a530f3e0f4b3b6e9be843fea401f"
server_key_iv: "63ea44c52d30963588b10fa3110c685b"
server_key_version: "encryptionkey@0"
I20260808 10:34:58.335826 7048 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:34:58.338353 7063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:58.339658 7048 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:34:58.339802 7048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "6faa9f501ce5474cbdec7d05706e88eb"
format_stamp: "Formatted at 2026-08-08 10:34:58 on dist-test-slave-9gcw"
server_key: "1249a530f3e0f4b3b6e9be843fea401f"
server_key_iv: "63ea44c52d30963588b10fa3110c685b"
server_key_version: "encryptionkey@0"
I20260808 10:34:58.339923 7048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:58.358006 7048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:58.361665 7048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:58.361899 7048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:58.370086 7048 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:35407
I20260808 10:34:58.370124 7134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:35407 every 8 connection(s)
I20260808 10:34:58.371066 7048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:34:58.371716 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 7048
I20260808 10:34:58.371904 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:34:58.372251 31306 external_mini_cluster.cc:1428] Setting key 38638f1ad9cade999cc394ae15c06a35
I20260808 10:34:58.374305 7135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:58.381302 7135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb: Bootstrap starting.
Aug 08 10:34:58 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:58.383853 7135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:58.384975 7135 log.cc:824] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:58.387194 7135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb: No bootstrap required, opened a new log
I20260808 10:34:58.387899 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:58.373717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60686 (local address 127.30.146.190:35407)
0808 10:34:58.374151 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:58.374167 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:58.374204 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:34:58.374935 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:58.374951 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:58.375278 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:58.375690 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:58.376784 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.377972 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.379172 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.379651 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.379772 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:58.382719 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:58.382727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:58.382745 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:58.382787 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:58.384624 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:58.385107 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:58.385113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:58.385190 (+ 77us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:58.385322 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:58.385632 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:58.385635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:58.385637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:58.386178 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:58.386413 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:58.386728 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 10:34:58.387050 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":144,"threads_started":1}
I20260808 10:34:58.390874 7135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faa9f501ce5474cbdec7d05706e88eb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35407 } }
I20260808 10:34:58.391129 7135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:58.391215 7135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6faa9f501ce5474cbdec7d05706e88eb, State: Initialized, Role: FOLLOWER
I20260808 10:34:58.391644 7135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faa9f501ce5474cbdec7d05706e88eb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35407 } }
I20260808 10:34:58.391796 7135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:34:58.391885 7135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:34:58.392017 7135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:58.393038 7135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faa9f501ce5474cbdec7d05706e88eb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35407 } }
I20260808 10:34:58.393412 7135 leader_election.cc:302] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6faa9f501ce5474cbdec7d05706e88eb; no voters:
I20260808 10:34:58.393746 7135 leader_election.cc:288] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:34:58.393915 7141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:58.394196 7141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [term 1 LEADER]: Becoming Leader. State: Replica: 6faa9f501ce5474cbdec7d05706e88eb, State: Running, Role: LEADER
I20260808 10:34:58.394511 7141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faa9f501ce5474cbdec7d05706e88eb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35407 } }
I20260808 10:34:58.395105 7135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:34:58.396070 7142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6faa9f501ce5474cbdec7d05706e88eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faa9f501ce5474cbdec7d05706e88eb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35407 } } }
I20260808 10:34:58.396209 7142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:58.396533 7143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6faa9f501ce5474cbdec7d05706e88eb. Latest consensus state: current_term: 1 leader_uuid: "6faa9f501ce5474cbdec7d05706e88eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faa9f501ce5474cbdec7d05706e88eb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35407 } } }
I20260808 10:34:58.396643 7143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb [sys.catalog]: This master's current role is: LEADER
I20260808 10:34:58.397009 7150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:34:58.399777 7150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:34:58.405017 7150 catalog_manager.cc:1347] Generated new cluster ID: 336ac4a524af49969ae273c8a28412b5
I20260808 10:34:58.405084 7150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:34:58.421715 7150 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:34:58.423033 7150 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:34:58.435235 7150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb: Generated new TSK 0
I20260808 10:34:58.436141 7150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab.
Entry for principal oryx/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab.
I20260808 10:34:58.477932 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:35407
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab
--principal=oryx/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:58.607775 7164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:58.608062 7164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:58.608108 7164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:58.613407 7164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:58.613500 7164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:58.613581 7164 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:34:58.613662 7164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:34:58.619445 7164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab
--principal=oryx/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.146.190:35407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:58.620582 7164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:58.621730 7164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:58.631726 7175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:58.631728 7172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:58.631992 7171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:58.632427 7164 server_base.cc:1034] running on GCE node
I20260808 10:34:58.632858 7164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:58.633733 7164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:58.634905 7164 hybrid_clock.cc:648] HybridClock initialized: now 1786185298634876 us; error 41 us; skew 500 ppm
Aug 08 10:34:58 dist-test-slave-9gcw krb5kdc[7033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185298, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:58.638199 7164 init.cc:377] Logged in from keytab as oryx/127.30.146.129@KRBTEST.COM (short username oryx)
I20260808 10:34:58.639214 7164 webserver.cc:460] Webserver started at http://127.30.146.129:36099/ using document root <none> and password file <none>
I20260808 10:34:58.639755 7164 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:58.639818 7164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:58.639990 7164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:58.642155 7164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "2f432dfdfa644d4181f1ac56b2e97873"
format_stamp: "Formatted at 2026-08-08 10:34:58 on dist-test-slave-9gcw"
server_key: "fab5d37d45cfb92938d6467498bdea02"
server_key_iv: "3e949ae51552435d2feba6a34d60080d"
server_key_version: "encryptionkey@0"
I20260808 10:34:58.642685 7164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "2f432dfdfa644d4181f1ac56b2e97873"
format_stamp: "Formatted at 2026-08-08 10:34:58 on dist-test-slave-9gcw"
server_key: "fab5d37d45cfb92938d6467498bdea02"
server_key_iv: "3e949ae51552435d2feba6a34d60080d"
server_key_version: "encryptionkey@0"
I20260808 10:34:58.646732 7164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:34:58.649207 7183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:58.650571 7164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:34:58.650823 7164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "2f432dfdfa644d4181f1ac56b2e97873"
format_stamp: "Formatted at 2026-08-08 10:34:58 on dist-test-slave-9gcw"
server_key: "fab5d37d45cfb92938d6467498bdea02"
server_key_iv: "3e949ae51552435d2feba6a34d60080d"
server_key_version: "encryptionkey@0"
I20260808 10:34:58.650951 7164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:58.666802 7164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:58.669989 7164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:58.670229 7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:58.671797 7164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:58.671870 7164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:58.671947 7164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:58.672003 7164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:34:58 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185298, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:58.683723 7164 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:33747
I20260808 10:34:58.683807 7310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:33747 every 8 connection(s)
I20260808 10:34:58.684839 7164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:34:58.686131 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 7164
I20260808 10:34:58.686265 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:34:58.686551 31306 external_mini_cluster.cc:1428] Setting key d09ff9576fe5930312fc6c5eb297c028
I20260808 10:34:58.686564 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:58.672567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:34337 (local address 127.30.146.190:35407)
0808 10:34:58.672698 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:58.672702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:58.673985 (+ 1283us) server_negotiation.cc:407] Connection header received
0808 10:34:58.675020 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:58.675024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:58.675092 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:58.675154 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:58.676896 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.677661 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.678838 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.679159 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.679267 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:58.682396 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:58.682400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:58.682403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:58.682434 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:58.684191 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:58.684689 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:58.684692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:58.684694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:58.684789 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:58.685247 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:58.685253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:58.685257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:58.685516 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:58.685616 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:58.686207 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 10:34:58.686392 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:34:58.687417 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:58.673173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35407 (local address 127.30.146.129:34337)
0808 10:34:58.673719 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:58.673772 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:58.674762 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:58.675595 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:58.675602 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:58.676072 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:58.676727 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:58.676745 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.677816 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.677819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:58.678704 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:58.678714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.679340 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.679344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:58.679577 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:58.680263 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:58.680290 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:58.682210 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:58.684315 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:58.684320 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:58.684331 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:58.684583 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:58.684917 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:58.684920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:58.684922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:58.685117 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:58.685672 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:58.685678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:58.685949 (+ 271us) client_negotiation.cc:769] Sending connection context
0808 10:34:58.686143 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 10:34:58.686484 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":143,"threads_started":1}
I20260808 10:34:58.688573 7312 heartbeater.cc:347] Connected to a master server at 127.30.146.190:35407
I20260808 10:34:58.688709 7312 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:58.689122 7312 heartbeater.cc:511] Master 127.30.146.190:35407 requested a full tablet report, sending...
I20260808 10:34:58.690707 7081 ts_manager.cc:194] Registered new tserver with Master: 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747)
I20260808 10:34:58.692269 7081 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:34337
WARNING: no policy specified for oryx/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:34:58.699747 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:58.691140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:49487 (local address 127.30.146.190:35407)
0808 10:34:58.691268 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:58.691272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:58.691495 (+ 223us) server_negotiation.cc:407] Connection header received
0808 10:34:58.691610 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:58.691613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:58.691653 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:58.691756 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:58.692190 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.693023 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.694208 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.694581 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.694709 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:58.696136 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:58.696143 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:58.696147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:58.696180 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:58.698051 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:58.698514 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:58.698519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:58.698522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:58.698616 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:58.698902 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:58.698905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:58.698907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:58.699167 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:58.699273 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:58.699522 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 10:34:58.699620 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":46}
I20260808 10:34:58.699753 7320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:58.691082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35407 (local address 127.30.146.129:49487)
0808 10:34:58.691409 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:58.691421 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:58.691532 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:58.691762 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:58.691765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:58.691971 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:58.692073 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:58.692079 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.693196 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.693200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:58.694065 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:58.694075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.694703 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.694705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:58.694775 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:58.695431 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:58.695449 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:58.695955 (+ 506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:58.698179 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:58.698181 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:58.698183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:58.698398 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:58.698719 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:58.698722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:58.698723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:58.698807 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:58.699258 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:58.699260 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:58.699345 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 10:34:58.699440 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:34:58.699555 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":5,"thread_start_us":100,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab.
Entry for principal oryx/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab.
I20260808 10:34:58.720098 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:35407
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab
--principal=oryx/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:58.837855 7323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:58.838184 7323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:58.838267 7323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:58.842379 7323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:58.842447 7323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:58.842487 7323 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:34:58.842549 7323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:34:58.848523 7323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab
--principal=oryx/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.146.190:35407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:58.849794 7323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:58.850865 7323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:58.861716 7328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:58.861730 7329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:58.861742 7331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:58.862363 7323 server_base.cc:1034] running on GCE node
I20260808 10:34:58.863077 7323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:58.863901 7323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:58.865129 7323 hybrid_clock.cc:648] HybridClock initialized: now 1786185298865092 us; error 50 us; skew 500 ppm
Aug 08 10:34:58 dist-test-slave-9gcw krb5kdc[7033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185298, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:58.868126 7323 init.cc:377] Logged in from keytab as oryx/127.30.146.130@KRBTEST.COM (short username oryx)
I20260808 10:34:58.868980 7323 webserver.cc:460] Webserver started at http://127.30.146.130:46171/ using document root <none> and password file <none>
I20260808 10:34:58.869586 7323 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:58.869652 7323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:58.869823 7323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:58.871920 7323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae"
format_stamp: "Formatted at 2026-08-08 10:34:58 on dist-test-slave-9gcw"
server_key: "6370ce836f976550646b5c46d7411e45"
server_key_iv: "a493192926ed45f66b2fda66c73a27fb"
server_key_version: "encryptionkey@0"
I20260808 10:34:58.872452 7323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae"
format_stamp: "Formatted at 2026-08-08 10:34:58 on dist-test-slave-9gcw"
server_key: "6370ce836f976550646b5c46d7411e45"
server_key_iv: "a493192926ed45f66b2fda66c73a27fb"
server_key_version: "encryptionkey@0"
I20260808 10:34:58.876403 7323 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:34:58.879165 7339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:58.880714 7323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:58.880846 7323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae"
format_stamp: "Formatted at 2026-08-08 10:34:58 on dist-test-slave-9gcw"
server_key: "6370ce836f976550646b5c46d7411e45"
server_key_iv: "a493192926ed45f66b2fda66c73a27fb"
server_key_version: "encryptionkey@0"
I20260808 10:34:58.880975 7323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:58.895957 7323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:58.899461 7323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:58.899672 7323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:58.901113 7323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:58.901219 7323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:58.901305 7323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:58.901360 7323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:34:58 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185298, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:58.913196 7323 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:41953
I20260808 10:34:58.914024 7471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:41953 every 8 connection(s)
I20260808 10:34:58.914220 7323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:34:58.916067 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:58.902311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:58399 (local address 127.30.146.190:35407)
0808 10:34:58.902478 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:58.902484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:58.903425 (+ 941us) server_negotiation.cc:407] Connection header received
0808 10:34:58.904799 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:58.904806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:58.904875 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:58.904978 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:58.906428 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.906979 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.908288 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.908601 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.908742 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:58.911948 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:58.911951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:58.911953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:58.911981 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:58.913812 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:58.914424 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:58.914427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:58.914428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:58.914521 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:58.914988 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:58.914992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:58.914994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:58.915217 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:58.915305 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:58.915835 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 10:34:58.915944 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:34:58.916956 7351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:58.902729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35407 (local address 127.30.146.130:58399)
0808 10:34:58.903247 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:58.903302 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:58.904483 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:58.905221 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:58.905230 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:58.905664 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:58.906252 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:58.906267 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.907131 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.907136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:58.908146 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:58.908157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.908798 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.908804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:58.909044 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:58.909821 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:58.909840 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:58.911754 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:58.913939 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:58.913944 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:58.913958 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:58.914312 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:58.914660 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:58.914666 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:58.914671 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:58.914859 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:58.915345 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:58.915351 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:58.915586 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 10:34:58.915762 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 10:34:58.916023 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":157,"threads_started":1}
I20260808 10:34:58.918155 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 7323
I20260808 10:34:58.918324 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:34:58.918395 7472 heartbeater.cc:347] Connected to a master server at 127.30.146.190:35407
I20260808 10:34:58.918635 7472 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:58.918931 31306 external_mini_cluster.cc:1428] Setting key 495ae4a945bd4f7a4e41766cfd6b346f
I20260808 10:34:58.919260 7472 heartbeater.cc:511] Master 127.30.146.190:35407 requested a full tablet report, sending...
I20260808 10:34:58.920394 7081 ts_manager.cc:194] Registered new tserver with Master: 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953)
I20260808 10:34:58.921128 7081 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:58399
I20260808 10:34:58.931341 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:58.920968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:45191 (local address 127.30.146.190:35407)
0808 10:34:58.921069 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:58.921073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:58.921410 (+ 337us) server_negotiation.cc:407] Connection header received
0808 10:34:58.921864 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:58.921867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:58.921916 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:58.921999 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:58.922437 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.923188 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.924142 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.924560 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.924691 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:58.926573 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:58.926577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:58.926579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:58.926641 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:58.928672 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:58.929379 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:58.929385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:58.929390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:58.929502 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:58.929936 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:58.929943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:58.929947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:58.930328 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:58.930443 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:58.930893 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 10:34:58.931048 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:34:58.931998 7479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:58.920905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35407 (local address 127.30.146.130:45191)
0808 10:34:58.921305 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:58.921321 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:58.921746 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:58.922016 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:58.922018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:58.922228 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:58.922326 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:58.922332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.923350 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.923354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:58.924010 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:58.924020 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:58.924879 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:58.924882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:58.924975 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:58.925746 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:58.925764 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:58.926367 (+ 603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:58.928843 (+ 2476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:58.928849 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:58.928854 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:58.929228 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:58.929651 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:58.929656 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:58.929659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:58.929768 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:58.930521 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:58.930526 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:58.930646 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 10:34:58.931536 (+ 890us) client_negotiation.cc:240] Negotiation successful
0808 10:34:58.931742 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":179,"threads_started":1}
WARNING: no policy specified for oryx/127.30.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab.
Entry for principal oryx/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab.
I20260808 10:34:58.956938 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:35407
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab
--principal=oryx/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:34:59.076830 7482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:34:59.077116 7482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:34:59.077234 7482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:34:59.081436 7482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:34:59.081518 7482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:34:59.081593 7482 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:34:59.081689 7482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:34:59.087410 7482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab
--principal=oryx/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.146.190:35407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:34:59.088601 7482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:34:59.089805 7482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:34:59.100050 7488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:59.100059 7490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:34:59.100150 7487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:34:59.100329 7482 server_base.cc:1034] running on GCE node
I20260808 10:34:59.100769 7482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:34:59.101459 7482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:34:59.102650 7482 hybrid_clock.cc:648] HybridClock initialized: now 1786185299102615 us; error 46 us; skew 500 ppm
Aug 08 10:34:59 dist-test-slave-9gcw krb5kdc[7033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185299, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:34:59.105551 7482 init.cc:377] Logged in from keytab as oryx/127.30.146.131@KRBTEST.COM (short username oryx)
I20260808 10:34:59.106709 7482 webserver.cc:460] Webserver started at http://127.30.146.131:37113/ using document root <none> and password file <none>
I20260808 10:34:59.107569 7482 fs_manager.cc:359] Metadata directory not provided
I20260808 10:34:59.107683 7482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:34:59.107955 7482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:34:59.111411 7482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "28e88b8ed6684731b6275e5e45c5d503"
format_stamp: "Formatted at 2026-08-08 10:34:59 on dist-test-slave-9gcw"
server_key: "f7a7cdeaa640be0beb04b551022ea6a2"
server_key_iv: "a0d6acbadd9cc973d61da19a8e11ded4"
server_key_version: "encryptionkey@0"
I20260808 10:34:59.112064 7482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "28e88b8ed6684731b6275e5e45c5d503"
format_stamp: "Formatted at 2026-08-08 10:34:59 on dist-test-slave-9gcw"
server_key: "f7a7cdeaa640be0beb04b551022ea6a2"
server_key_iv: "a0d6acbadd9cc973d61da19a8e11ded4"
server_key_version: "encryptionkey@0"
I20260808 10:34:59.116348 7482 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:34:59.119364 7497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:59.120770 7482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:34:59.120903 7482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "28e88b8ed6684731b6275e5e45c5d503"
format_stamp: "Formatted at 2026-08-08 10:34:59 on dist-test-slave-9gcw"
server_key: "f7a7cdeaa640be0beb04b551022ea6a2"
server_key_iv: "a0d6acbadd9cc973d61da19a8e11ded4"
server_key_version: "encryptionkey@0"
I20260808 10:34:59.121028 7482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:34:59.143764 7482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:34:59.147562 7482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:34:59.147785 7482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:34:59.149492 7482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:34:59.149571 7482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:34:59.149641 7482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:34:59.149665 7482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:34:59 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185299, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:59.160961 7482 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:45993
I20260808 10:34:59.161041 7628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:45993 every 8 connection(s)
I20260808 10:34:59.162444 7482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:34:59.164788 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.150270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:47751 (local address 127.30.146.190:35407)
0808 10:34:59.150433 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.150438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.151462 (+ 1024us) server_negotiation.cc:407] Connection header received
0808 10:34:59.152810 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.152815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.152879 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.153004 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:59.154447 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.155400 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.156722 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.157136 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.157307 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.160875 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:59.160879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:59.160881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:59.160912 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:59.162572 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:59.163175 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:59.163179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:59.163181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:59.163281 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:59.163639 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:59.163643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:59.163645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:59.163852 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:59.163966 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.164468 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.164599 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:34:59.165324 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 7482
I20260808 10:34:59.165473 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:34:59.165822 31306 external_mini_cluster.cc:1428] Setting key dd8de7c08c6a9421c12e9f7b28048c88
I20260808 10:34:59.165855 7509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.150762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35407 (local address 127.30.146.131:47751)
0808 10:34:59.151271 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.151336 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.152525 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.153242 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.153250 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.153577 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:59.154265 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.154289 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.155553 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.155557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.156548 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.156558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.157350 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.157355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.157584 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.158922 (+ 1338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:59.158946 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:59.160670 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:59.162683 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:59.162688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:59.162702 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:59.163048 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:59.163375 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:59.163378 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:59.163380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:59.163529 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:59.163965 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:59.163972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:59.164224 (+ 252us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.164422 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.164735 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":111,"threads_started":1}
I20260808 10:34:59.167208 7629 heartbeater.cc:347] Connected to a master server at 127.30.146.190:35407
I20260808 10:34:59.167369 7629 heartbeater.cc:464] Registering TS with master...
I20260808 10:34:59.167922 7629 heartbeater.cc:511] Master 127.30.146.190:35407 requested a full tablet report, sending...
I20260808 10:34:59.168963 7080 ts_manager.cc:194] Registered new tserver with Master: 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131:45993)
I20260808 10:34:59.169783 7080 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:47751
I20260808 10:34:59.176407 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.169396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:47651 (local address 127.30.146.190:35407)
0808 10:34:59.169523 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.169527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.169750 (+ 223us) server_negotiation.cc:407] Connection header received
0808 10:34:59.169828 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.169831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.169923 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.170056 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:59.170588 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.171087 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.171911 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.172139 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.172250 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.173424 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:59.173428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:59.173430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:59.173456 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:59.174775 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:59.175203 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:59.175208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:59.175210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:59.175295 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:59.175590 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:59.175595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:59.175597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:59.175777 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:59.175885 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.176148 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.176269 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:34:59.176419 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.169328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35407 (local address 127.30.146.131:47651)
0808 10:34:59.169630 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.169644 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.169749 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.170090 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.170096 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.170337 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:59.170465 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.170473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.171215 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.171218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.171786 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.171794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.172257 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.172261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.172353 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.172912 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:59.172923 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:59.173294 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:59.174893 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:59.174896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:59.174898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:59.175100 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:59.175404 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:59.175407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:59.175410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:59.175499 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:59.175877 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:59.175880 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:59.175981 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.176092 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.176238 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":89,"threads_started":1}
I20260808 10:34:59.180013 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:34:59 dist-test-slave-9gcw krb5kdc[7033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185299, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:34:59 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:34:59 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:34:59.206357 7139 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:34:59.199952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.30.146.190:35407)
0808 10:34:59.200114 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.200118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.200130 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:34:59.200172 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.200175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.200213 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.200329 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:59.201075 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.201653 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.202530 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.202755 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.202868 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.206227 (+ 3359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60696: BlockingRecv error: recv got EOF from 127.0.0.1:60696 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 10:34:59 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185299, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:59.216586 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.208062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.30.146.190:35407)
0808 10:34:59.208188 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.208194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.208270 (+ 76us) server_negotiation.cc:407] Connection header received
0808 10:34:59.208367 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.208369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.208411 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.208495 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:59.209065 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.209701 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.210690 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.210909 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.210998 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.213284 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:59.213287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:59.213289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:59.213312 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:59.215059 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:59.215461 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:59.215464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:59.215465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:59.215543 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:59.215884 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:59.215886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:59.215887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:59.216065 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:59.216184 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.216367 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.216459 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:34:59.219825 7080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:34:59.223194 7080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:59.238255 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.232945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32846 (local address 127.30.146.131:45993)
0808 10:34:59.233388 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.233400 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.233618 (+ 218us) server_negotiation.cc:407] Connection header received
0808 10:34:59.233699 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.233704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.233896 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.234063 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:59.234422 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.235393 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.236792 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.237539 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.237683 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.237818 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.237989 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.238098 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":142,"threads_started":1}
I20260808 10:34:59.238236 7654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.232101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:45993 (local address 127.0.0.1:32846)
0808 10:34:59.233421 (+ 1320us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.233436 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.233527 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.234039 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.234042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.234060 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:59.234170 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.234179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.235515 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.235518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.236663 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.236672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.237666 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.237670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.237788 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.237831 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.237947 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.238071 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1253,"thread_start_us":94,"threads_started":1}
I20260808 10:34:59.238739 7655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.232198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35838 (local address 127.30.146.130:41953)
0808 10:34:59.232728 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.232741 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.232864 (+ 123us) server_negotiation.cc:407] Connection header received
0808 10:34:59.233088 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.233096 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.233387 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.233564 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:59.233898 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.235110 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.236658 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.237534 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.237682 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.237818 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.238458 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.238589 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":110,"threads_started":1}
I20260808 10:34:59.238772 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.232727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:33747 (local address 127.0.0.1:44014)
0808 10:34:59.233883 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.233896 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.233977 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.234621 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.234624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.234643 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:59.234741 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.234747 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.236005 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.236008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.237215 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.237225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.238367 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.238371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.238478 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.238501 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.238575 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.238634 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":35,"thread_start_us":96,"threads_started":1}
I20260808 10:34:59.238893 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.233212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44014 (local address 127.30.146.129:33747)
0808 10:34:59.234221 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.234228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.234252 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:34:59.234323 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.234328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.234487 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.234623 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:59.234865 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.235869 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.237343 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.238052 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.238165 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.238299 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.238627 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.238720 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":900,"thread_start_us":65,"threads_started":1}
I20260808 10:34:59.240325 7652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.232032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41953 (local address 127.0.0.1:35838)
0808 10:34:59.232708 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.232759 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.232902 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.233566 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.233572 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.233593 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:59.233749 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.233757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.235229 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.235232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.236510 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.236519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.237892 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.237895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.238004 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.238027 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.238114 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.238180 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260808 10:34:59.241619 7554 tablet_service.cc:1467] Processing CreateTablet for tablet 9f57362c352f4aadbdf13ddc5e38544c (DEFAULT_TABLE table=test-table [id=2c1f67af96184d0c90464a6b7b1f0686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:59.241829 7237 tablet_service.cc:1467] Processing CreateTablet for tablet 9f57362c352f4aadbdf13ddc5e38544c (DEFAULT_TABLE table=test-table [id=2c1f67af96184d0c90464a6b7b1f0686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:59.242657 7554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f57362c352f4aadbdf13ddc5e38544c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:59.242871 7237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f57362c352f4aadbdf13ddc5e38544c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:59.242910 7398 tablet_service.cc:1467] Processing CreateTablet for tablet 9f57362c352f4aadbdf13ddc5e38544c (DEFAULT_TABLE table=test-table [id=2c1f67af96184d0c90464a6b7b1f0686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:59.244052 7398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f57362c352f4aadbdf13ddc5e38544c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:59.249748 7659 tablet_bootstrap.cc:492] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: Bootstrap starting.
I20260808 10:34:59.251395 7663 tablet_bootstrap.cc:492] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Bootstrap starting.
I20260808 10:34:59.252533 7659 tablet_bootstrap.cc:654] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:59.253456 7662 tablet_bootstrap.cc:492] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: Bootstrap starting.
I20260808 10:34:59.253492 7659 log.cc:824] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:59.253551 7663 tablet_bootstrap.cc:654] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:59.254387 7663 log.cc:824] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:59.255712 7659 tablet_bootstrap.cc:492] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: No bootstrap required, opened a new log
I20260808 10:34:59.255841 7662 tablet_bootstrap.cc:654] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:59.255909 7659 ts_tablet_manager.cc:1397] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:34:59.256464 7663 tablet_bootstrap.cc:492] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: No bootstrap required, opened a new log
I20260808 10:34:59.256533 7662 log.cc:824] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: Log is configured to *not* fsync() on all Append() calls
I20260808 10:34:59.256652 7663 ts_tablet_manager.cc:1397] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:34:59.258265 7662 tablet_bootstrap.cc:492] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: No bootstrap required, opened a new log
I20260808 10:34:59.258457 7662 ts_tablet_manager.cc:1397] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:34:59.260154 7659 raft_consensus.cc:348] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } }
I20260808 10:34:59.260434 7659 raft_consensus.cc:374] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:59.260486 7659 raft_consensus.cc:729] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28e88b8ed6684731b6275e5e45c5d503, State: Initialized, Role: FOLLOWER
I20260808 10:34:59.260547 7663 raft_consensus.cc:348] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } }
I20260808 10:34:59.260777 7663 raft_consensus.cc:374] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:59.260825 7663 raft_consensus.cc:729] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbb9e566f3d4b22bcdcfc1e2b21d5ae, State: Initialized, Role: FOLLOWER
I20260808 10:34:59.260990 7659 consensus_queue.cc:260] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } }
I20260808 10:34:59.261276 7663 consensus_queue.cc:260] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } }
I20260808 10:34:59.262022 7659 ts_tablet_manager.cc:1428] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260808 10:34:59.262076 7663 ts_tablet_manager.cc:1428] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:34:59.262210 7662 raft_consensus.cc:348] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } }
I20260808 10:34:59.262415 7629 heartbeater.cc:503] Master 127.30.146.190:35407 was elected leader, sending a full tablet report...
I20260808 10:34:59.262444 7472 heartbeater.cc:503] Master 127.30.146.190:35407 was elected leader, sending a full tablet report...
I20260808 10:34:59.262476 7662 raft_consensus.cc:374] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:59.262522 7662 raft_consensus.cc:729] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f432dfdfa644d4181f1ac56b2e97873, State: Initialized, Role: FOLLOWER
I20260808 10:34:59.262908 7662 consensus_queue.cc:260] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } }
I20260808 10:34:59.263535 7312 heartbeater.cc:503] Master 127.30.146.190:35407 was elected leader, sending a full tablet report...
I20260808 10:34:59.263803 7662 ts_tablet_manager.cc:1428] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260808 10:34:59.413699 7630 tablet.cc:2369] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:34:59.415519 7473 tablet.cc:2369] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:34:59.436432 7314 tablet.cc:2369] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:34:59.468206 7667 raft_consensus.cc:482] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:59.468679 7667 raft_consensus.cc:504] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } }
I20260808 10:34:59.469945 7667 leader_election.cc:288] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953), 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747)
I20260808 10:34:59.474025 7509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.470298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41953 (local address 127.30.146.131:42215)
0808 10:34:59.470451 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.470468 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.470586 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.471023 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.471025 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.471047 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:59.471179 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.471188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.472017 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.472020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.473005 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.473015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.473713 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.473716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.473795 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.473816 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.473870 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.473923 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 10:34:59.474093 7655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.470365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:42215 (local address 127.30.146.130:41953)
0808 10:34:59.470518 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.470522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.470537 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:34:59.470850 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.470854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.470916 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.470983 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:59.471294 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.471908 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.473129 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.473599 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.473728 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.473797 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.473896 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.473969 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:34:59.474889 7670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.470591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:33747 (local address 127.30.146.131:55023)
0808 10:34:59.471028 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.471040 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.471142 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.471429 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.471431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.471446 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:59.471819 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.471826 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.472579 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.472582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.473528 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.473537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.474433 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.474437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.474541 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.474558 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.474674 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.474750 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":159,"threads_started":1}
I20260808 10:34:59.474877 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.470773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:55023 (local address 127.30.146.129:33747)
0808 10:34:59.470908 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.470912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.471133 (+ 221us) server_negotiation.cc:407] Connection header received
0808 10:34:59.471277 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.471280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.471335 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.471413 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:59.471948 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.472470 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.473652 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.474284 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.474437 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.474487 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.474695 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.474768 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:34:59.474841 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f57362c352f4aadbdf13ddc5e38544c" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" is_pre_election: true
I20260808 10:34:59.475189 7425 raft_consensus.cc:2454] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 0.
I20260808 10:34:59.475697 7498 leader_election.cc:302] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e88b8ed6684731b6275e5e45c5d503, 6bbb9e566f3d4b22bcdcfc1e2b21d5ae; no voters:
I20260808 10:34:59.475766 7262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f57362c352f4aadbdf13ddc5e38544c" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f432dfdfa644d4181f1ac56b2e97873" is_pre_election: true
I20260808 10:34:59.475991 7667 raft_consensus.cc:2781] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:59.476094 7667 raft_consensus.cc:482] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:59.476066 7262 raft_consensus.cc:2454] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 0.
I20260808 10:34:59.476156 7667 raft_consensus.cc:3037] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:59.477337 7667 raft_consensus.cc:504] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } }
I20260808 10:34:59.477761 7667 leader_election.cc:288] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 election: Requested vote from peers 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953), 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747)
I20260808 10:34:59.478140 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f57362c352f4aadbdf13ddc5e38544c" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae"
I20260808 10:34:59.478266 7262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f57362c352f4aadbdf13ddc5e38544c" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f432dfdfa644d4181f1ac56b2e97873"
I20260808 10:34:59.478297 7425 raft_consensus.cc:3037] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:59.478413 7262 raft_consensus.cc:3037] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:59.479382 7425 raft_consensus.cc:2454] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 1.
I20260808 10:34:59.479540 7262 raft_consensus.cc:2454] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 1.
I20260808 10:34:59.479712 7498 leader_election.cc:302] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e88b8ed6684731b6275e5e45c5d503, 6bbb9e566f3d4b22bcdcfc1e2b21d5ae; no voters:
I20260808 10:34:59.480036 7667 raft_consensus.cc:2781] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:59.480279 7667 raft_consensus.cc:686] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 1 LEADER]: Becoming Leader. State: Replica: 28e88b8ed6684731b6275e5e45c5d503, State: Running, Role: LEADER
I20260808 10:34:59.480613 7667 consensus_queue.cc:237] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } }
I20260808 10:34:59.484081 7080 catalog_manager.cc:5495] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131). New cstate: current_term: 1 leader_uuid: "28e88b8ed6684731b6275e5e45c5d503" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:59.531744 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.528146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.30.146.131:45993)
0808 10:34:59.528294 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.528299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.528316 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:34:59.528409 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.528411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.528466 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.528578 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:59.528829 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.529409 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.530272 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.530492 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.530599 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.530905 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:59.531340 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:59.531467 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.531526 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.531585 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:34:59.537786 7262 raft_consensus.cc:1261] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Refusing update from remote peer 28e88b8ed6684731b6275e5e45c5d503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:59.537786 7425 raft_consensus.cc:1261] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Refusing update from remote peer 28e88b8ed6684731b6275e5e45c5d503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:59.538506 7667 consensus_queue.cc:1035] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:59.538724 7671 consensus_queue.cc:1035] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:59.546888 7675 mvcc.cc:205] Tried to move back new op lower bound from 7316214986899480576 to 7316214986675294208. Current Snapshot: MvccSnapshot[applied={T|T < 7316214986899480576}]
I20260808 10:34:59.549609 7676 mvcc.cc:205] Tried to move back new op lower bound from 7316214986899480576 to 7316214986675294208. Current Snapshot: MvccSnapshot[applied={T|T < 7316214986899480576}]
I20260808 10:34:59.556380 7082 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:59.556720 7082 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:34:59.559685 7082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb: Sending DeleteTablet for 3 replicas of tablet 9f57362c352f4aadbdf13ddc5e38544c
I20260808 10:34:59.560683 7554 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f57362c352f4aadbdf13ddc5e38544c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:59 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:32846
I20260808 10:34:59.560722 7237 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f57362c352f4aadbdf13ddc5e38544c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:59 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:44014
I20260808 10:34:59.560683 7398 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f57362c352f4aadbdf13ddc5e38544c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:34:59 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:35838
I20260808 10:34:59.561216 7687 tablet_replica.cc:330] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: stopping tablet replica
I20260808 10:34:59.561401 7688 tablet_replica.cc:330] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: stopping tablet replica
I20260808 10:34:59.561564 7687 raft_consensus.cc:2229] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:59.561750 7688 raft_consensus.cc:2229] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:34:59.561906 7687 raft_consensus.cc:2258] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:59.562080 7688 raft_consensus.cc:2258] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:59.562549 7689 tablet_replica.cc:330] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: stopping tablet replica
I20260808 10:34:59.562497 7082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:34:59.562891 7689 raft_consensus.cc:2229] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 1 LEADER]: Raft consensus shutting down.
W20260808 10:34:59.563426 7082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:59.563452 7687 ts_tablet_manager.cc:1905] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:59.563323 7689 raft_consensus.cc:2258] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:34:59.568740 7688 ts_tablet_manager.cc:1905] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:59.569439 7687 ts_tablet_manager.cc:1918] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:59.569653 7687 log.cc:1196] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/9f57362c352f4aadbdf13ddc5e38544c
I20260808 10:34:59.569908 7237 tablet_service.cc:1467] Processing CreateTablet for tablet ea99c162398c44c8b605d99968c5c8be (DEFAULT_TABLE table=test-table [id=f67d9f4e26f247f2bac5acd7b43215d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:59.570186 7237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea99c162398c44c8b605d99968c5c8be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:59.570322 7687 ts_tablet_manager.cc:1939] T 9f57362c352f4aadbdf13ddc5e38544c P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting consensus metadata
I20260808 10:34:59.572131 7688 ts_tablet_manager.cc:1918] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:59.572253 7688 log.cc:1196] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/9f57362c352f4aadbdf13ddc5e38544c
I20260808 10:34:59.572652 7688 ts_tablet_manager.cc:1939] T 9f57362c352f4aadbdf13ddc5e38544c P 2f432dfdfa644d4181f1ac56b2e97873: Deleting consensus metadata
I20260808 10:34:59.572685 7398 tablet_service.cc:1467] Processing CreateTablet for tablet ea99c162398c44c8b605d99968c5c8be (DEFAULT_TABLE table=test-table [id=f67d9f4e26f247f2bac5acd7b43215d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:59.572958 7398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea99c162398c44c8b605d99968c5c8be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:59.573275 7662 tablet_bootstrap.cc:492] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873: Bootstrap starting.
I20260808 10:34:59.573869 7689 ts_tablet_manager.cc:1905] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:34:59.574250 7662 tablet_bootstrap.cc:654] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:59.575819 7662 tablet_bootstrap.cc:492] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873: No bootstrap required, opened a new log
I20260808 10:34:59.575906 7662 ts_tablet_manager.cc:1397] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:59.576345 7662 raft_consensus.cc:348] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.576460 7662 raft_consensus.cc:374] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:59.576498 7662 raft_consensus.cc:729] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f432dfdfa644d4181f1ac56b2e97873, State: Initialized, Role: FOLLOWER
I20260808 10:34:59.576609 7662 consensus_queue.cc:260] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.576858 7662 ts_tablet_manager.cc:1428] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:59.576804 7064 catalog_manager.cc:4841] TS 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953): tablet 9f57362c352f4aadbdf13ddc5e38544c (table test-table [id=2c1f67af96184d0c90464a6b7b1f0686]) successfully deleted
I20260808 10:34:59.577421 7064 catalog_manager.cc:4841] TS 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747): tablet 9f57362c352f4aadbdf13ddc5e38544c (table test-table [id=2c1f67af96184d0c90464a6b7b1f0686]) successfully deleted
I20260808 10:34:59.577730 7689 ts_tablet_manager.cc:1918] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:34:59.577888 7689 log.cc:1196] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/9f57362c352f4aadbdf13ddc5e38544c
I20260808 10:34:59.578121 7554 tablet_service.cc:1467] Processing CreateTablet for tablet ea99c162398c44c8b605d99968c5c8be (DEFAULT_TABLE table=test-table [id=f67d9f4e26f247f2bac5acd7b43215d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:34:59.578385 7554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea99c162398c44c8b605d99968c5c8be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:59.579137 7663 tablet_bootstrap.cc:492] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Bootstrap starting.
I20260808 10:34:59.578393 7689 ts_tablet_manager.cc:1939] T 9f57362c352f4aadbdf13ddc5e38544c P 28e88b8ed6684731b6275e5e45c5d503: Deleting consensus metadata
I20260808 10:34:59.580386 7663 tablet_bootstrap.cc:654] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:59.580504 7659 tablet_bootstrap.cc:492] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503: Bootstrap starting.
I20260808 10:34:59.581693 7659 tablet_bootstrap.cc:654] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:59.581753 7663 tablet_bootstrap.cc:492] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: No bootstrap required, opened a new log
I20260808 10:34:59.581845 7663 ts_tablet_manager.cc:1397] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:59.582569 7663 raft_consensus.cc:348] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.582684 7663 raft_consensus.cc:374] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:59.582677 7067 catalog_manager.cc:4841] TS 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131:45993): tablet 9f57362c352f4aadbdf13ddc5e38544c (table test-table [id=2c1f67af96184d0c90464a6b7b1f0686]) successfully deleted
I20260808 10:34:59.582715 7663 raft_consensus.cc:729] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbb9e566f3d4b22bcdcfc1e2b21d5ae, State: Initialized, Role: FOLLOWER
I20260808 10:34:59.582917 7663 consensus_queue.cc:260] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.583559 7663 ts_tablet_manager.cc:1428] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:34:59.583655 7659 tablet_bootstrap.cc:492] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503: No bootstrap required, opened a new log
I20260808 10:34:59.583755 7659 ts_tablet_manager.cc:1397] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:59.584367 7659 raft_consensus.cc:348] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.584537 7659 raft_consensus.cc:374] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:59.584604 7659 raft_consensus.cc:729] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28e88b8ed6684731b6275e5e45c5d503, State: Initialized, Role: FOLLOWER
I20260808 10:34:59.584767 7659 consensus_queue.cc:260] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.585198 7659 ts_tablet_manager.cc:1428] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:34:59.639645 7682 raft_consensus.cc:482] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:59.639812 7682 raft_consensus.cc:504] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.640375 7682 leader_election.cc:288] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747), 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953)
I20260808 10:34:59.640918 7262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea99c162398c44c8b605d99968c5c8be" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f432dfdfa644d4181f1ac56b2e97873" is_pre_election: true
I20260808 10:34:59.640964 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea99c162398c44c8b605d99968c5c8be" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" is_pre_election: true
I20260808 10:34:59.641108 7262 raft_consensus.cc:2454] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 0.
I20260808 10:34:59.641105 7425 raft_consensus.cc:2454] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 0.
I20260808 10:34:59.641496 7498 leader_election.cc:302] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e88b8ed6684731b6275e5e45c5d503, 2f432dfdfa644d4181f1ac56b2e97873; no voters:
I20260808 10:34:59.641799 7682 raft_consensus.cc:2781] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:59.641872 7682 raft_consensus.cc:482] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:59.641916 7682 raft_consensus.cc:3037] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:59.643091 7682 raft_consensus.cc:504] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.643496 7682 leader_election.cc:288] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 election: Requested vote from peers 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747), 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953)
I20260808 10:34:59.643827 7262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea99c162398c44c8b605d99968c5c8be" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f432dfdfa644d4181f1ac56b2e97873"
I20260808 10:34:59.643954 7262 raft_consensus.cc:3037] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:59.643997 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea99c162398c44c8b605d99968c5c8be" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae"
I20260808 10:34:59.644107 7425 raft_consensus.cc:3037] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:59.645234 7262 raft_consensus.cc:2454] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 1.
I20260808 10:34:59.645287 7425 raft_consensus.cc:2454] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 1.
I20260808 10:34:59.645612 7498 leader_election.cc:302] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e88b8ed6684731b6275e5e45c5d503, 6bbb9e566f3d4b22bcdcfc1e2b21d5ae; no voters:
I20260808 10:34:59.645874 7682 raft_consensus.cc:2781] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:59.645978 7682 raft_consensus.cc:686] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 1 LEADER]: Becoming Leader. State: Replica: 28e88b8ed6684731b6275e5e45c5d503, State: Running, Role: LEADER
I20260808 10:34:59.646144 7682 consensus_queue.cc:237] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.648092 7082 catalog_manager.cc:5495] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131). New cstate: current_term: 1 leader_uuid: "28e88b8ed6684731b6275e5e45c5d503" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } health_report { overall_health: HEALTHY } } }
Aug 08 10:34:59 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185298, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.190@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:34:59.694456 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.682964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:35407 (local address 127.0.0.1:60712)
0808 10:34:59.683269 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.683287 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.683417 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.683878 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.683882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.684582 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:34:59.684736 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.684747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.685861 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.685865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.686743 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.686756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.687454 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.687457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.687534 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.688270 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:34:59.688285 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:34:59.690239 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:34:59.692301 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:59.692313 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:59.692320 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:59.692674 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:59.692987 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:34:59.692990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:34:59.692993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:34:59.693125 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:34:59.693650 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:34:59.693658 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:34:59.693808 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.693956 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.694160 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":263,"threads_started":1}
I20260808 10:34:59.694437 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.683056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60712 (local address 127.30.146.190:35407)
0808 10:34:59.683299 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.683304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.683388 (+ 84us) server_negotiation.cc:407] Connection header received
0808 10:34:59.683544 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.683551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.683611 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.683724 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:34:59.684888 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.685716 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.686888 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.687192 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.687302 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.690397 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:34:59.690401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:34:59.690404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:34:59.690427 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:34:59.692162 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:59.692784 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:59.692787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:59.692789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:59.692872 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:34:59.693289 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:34:59.693292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:34:59.693294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:34:59.693515 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:34:59.693690 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.694029 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.694227 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":89}
I20260808 10:34:59.702046 7082 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:60712:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:34:59.702986 7082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:34:59.708496 7237 tablet_service.cc:1467] Processing CreateTablet for tablet 3878cc3ae085443ab1ef028e12951226 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ab2327f6f79d4f83b0448e31040f1085]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:34:59.708611 7554 tablet_service.cc:1467] Processing CreateTablet for tablet 3878cc3ae085443ab1ef028e12951226 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ab2327f6f79d4f83b0448e31040f1085]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:34:59.708559 7398 tablet_service.cc:1467] Processing CreateTablet for tablet 3878cc3ae085443ab1ef028e12951226 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ab2327f6f79d4f83b0448e31040f1085]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:34:59.708961 7554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3878cc3ae085443ab1ef028e12951226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:59.708961 7398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3878cc3ae085443ab1ef028e12951226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:59.708968 7237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3878cc3ae085443ab1ef028e12951226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:34:59.712078 7659 tablet_bootstrap.cc:492] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: Bootstrap starting.
I20260808 10:34:59.712447 7662 tablet_bootstrap.cc:492] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: Bootstrap starting.
I20260808 10:34:59.713126 7659 tablet_bootstrap.cc:654] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:59.713454 7662 tablet_bootstrap.cc:654] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:59.714154 7663 tablet_bootstrap.cc:492] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Bootstrap starting.
I20260808 10:34:59.714854 7659 tablet_bootstrap.cc:492] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: No bootstrap required, opened a new log
I20260808 10:34:59.714960 7659 ts_tablet_manager.cc:1397] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 10:34:59.715202 7663 tablet_bootstrap.cc:654] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Neither blocks nor log segments found. Creating new log.
I20260808 10:34:59.715443 7659 raft_consensus.cc:348] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.715564 7659 raft_consensus.cc:374] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:59.715605 7659 raft_consensus.cc:729] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28e88b8ed6684731b6275e5e45c5d503, State: Initialized, Role: FOLLOWER
I20260808 10:34:59.715752 7659 consensus_queue.cc:260] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.715996 7682 tablet_replica.cc:439] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } }
I20260808 10:34:59.716147 7682 tablet_replica.cc:442] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:59.716696 7663 tablet_bootstrap.cc:492] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: No bootstrap required, opened a new log
I20260808 10:34:59.716814 7663 ts_tablet_manager.cc:1397] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:34:59.717104 7659 ts_tablet_manager.cc:1428] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 10:34:59.717335 7663 raft_consensus.cc:348] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.717479 7663 raft_consensus.cc:374] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:59.717530 7663 raft_consensus.cc:729] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbb9e566f3d4b22bcdcfc1e2b21d5ae, State: Initialized, Role: FOLLOWER
I20260808 10:34:59.717716 7663 consensus_queue.cc:260] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.718032 7662 tablet_bootstrap.cc:492] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: No bootstrap required, opened a new log
I20260808 10:34:59.718120 7662 ts_tablet_manager.cc:1397] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 10:34:59.718041 7668 tablet_replica.cc:439] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } }
I20260808 10:34:59.718178 7663 ts_tablet_manager.cc:1428] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:34:59.718220 7668 tablet_replica.cc:442] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:59.718552 7662 raft_consensus.cc:348] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.718636 7662 raft_consensus.cc:374] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:34:59.718688 7662 raft_consensus.cc:729] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f432dfdfa644d4181f1ac56b2e97873, State: Initialized, Role: FOLLOWER
I20260808 10:34:59.718814 7662 consensus_queue.cc:260] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.719111 7662 ts_tablet_manager.cc:1428] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:34:59.719149 7669 tablet_replica.cc:439] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } }
I20260808 10:34:59.719321 7669 tablet_replica.cc:442] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:59.891155 7669 raft_consensus.cc:482] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:34:59.891346 7669 raft_consensus.cc:504] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.892627 7669 leader_election.cc:288] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953), 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131:45993)
I20260808 10:34:59.897898 7655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.893032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:35387 (local address 127.30.146.130:41953)
0808 10:34:59.893239 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.893247 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.893345 (+ 98us) server_negotiation.cc:407] Connection header received
0808 10:34:59.893615 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.893619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.893679 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.893766 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:59.894157 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.894751 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.896498 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.897213 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.897355 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.897422 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.897656 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.897741 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 10:34:59.897917 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.893278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:48323 (local address 127.30.146.131:45993)
0808 10:34:59.893417 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.893421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.893512 (+ 91us) server_negotiation.cc:407] Connection header received
0808 10:34:59.893809 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.893812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.893858 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.893939 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:34:59.894381 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.895211 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.896722 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.897274 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.897455 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.897509 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.897695 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.897783 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:34:59.897924 7706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.892858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41953 (local address 127.30.146.129:35387)
0808 10:34:59.893240 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.893257 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.893440 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.893808 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.893811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.893831 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:59.893993 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.894001 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.894897 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.894901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.896362 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.896373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.897350 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.897354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.897488 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.897514 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.897638 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.897754 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":3,"thread_start_us":145,"threads_started":1}
I20260808 10:34:59.897943 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.893042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:45993 (local address 127.30.146.129:48323)
0808 10:34:59.893406 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.893427 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.893560 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.893965 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.893968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.893984 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:34:59.894137 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.894146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.895374 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.895377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.896506 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.896519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.897457 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.897460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.897560 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.897585 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.897676 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.897802 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":30,"thread_start_us":169,"threads_started":1}
I20260808 10:34:59.898717 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3878cc3ae085443ab1ef028e12951226" candidate_uuid: "2f432dfdfa644d4181f1ac56b2e97873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" is_pre_election: true
I20260808 10:34:59.898901 7425 raft_consensus.cc:2454] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f432dfdfa644d4181f1ac56b2e97873 in term 0.
I20260808 10:34:59.898926 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3878cc3ae085443ab1ef028e12951226" candidate_uuid: "2f432dfdfa644d4181f1ac56b2e97873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28e88b8ed6684731b6275e5e45c5d503" is_pre_election: true
I20260808 10:34:59.899200 7562 raft_consensus.cc:2454] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f432dfdfa644d4181f1ac56b2e97873 in term 0.
I20260808 10:34:59.899601 7187 leader_election.cc:302] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e88b8ed6684731b6275e5e45c5d503, 2f432dfdfa644d4181f1ac56b2e97873; no voters:
I20260808 10:34:59.899904 7669 raft_consensus.cc:2781] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:34:59.899968 7669 raft_consensus.cc:482] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:34:59.900008 7669 raft_consensus.cc:3037] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:59.901026 7669 raft_consensus.cc:504] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.901453 7669 leader_election.cc:288] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [CANDIDATE]: Term 1 election: Requested vote from peers 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953), 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131:45993)
I20260808 10:34:59.901832 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3878cc3ae085443ab1ef028e12951226" candidate_uuid: "2f432dfdfa644d4181f1ac56b2e97873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28e88b8ed6684731b6275e5e45c5d503"
I20260808 10:34:59.901829 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3878cc3ae085443ab1ef028e12951226" candidate_uuid: "2f432dfdfa644d4181f1ac56b2e97873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae"
I20260808 10:34:59.901940 7562 raft_consensus.cc:3037] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:59.901979 7425 raft_consensus.cc:3037] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:34:59.902995 7562 raft_consensus.cc:2454] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f432dfdfa644d4181f1ac56b2e97873 in term 1.
I20260808 10:34:59.903020 7425 raft_consensus.cc:2454] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f432dfdfa644d4181f1ac56b2e97873 in term 1.
I20260808 10:34:59.903326 7187 leader_election.cc:302] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e88b8ed6684731b6275e5e45c5d503, 2f432dfdfa644d4181f1ac56b2e97873; no voters:
I20260808 10:34:59.903501 7669 raft_consensus.cc:2781] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:34:59.903723 7669 raft_consensus.cc:686] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 LEADER]: Becoming Leader. State: Replica: 2f432dfdfa644d4181f1ac56b2e97873, State: Running, Role: LEADER
I20260808 10:34:59.904014 7669 consensus_queue.cc:237] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:34:59.905442 7709 tablet_replica.cc:439] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: TxnStatusTablet state changed. Reason: New leader 2f432dfdfa644d4181f1ac56b2e97873. Latest consensus state: current_term: 1 leader_uuid: "2f432dfdfa644d4181f1ac56b2e97873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } }
I20260808 10:34:59.905565 7709 tablet_replica.cc:442] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:34:59.905890 7711 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:34:59.906050 7711 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:34:59.906841 7082 catalog_manager.cc:5495] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129). New cstate: current_term: 1 leader_uuid: "2f432dfdfa644d4181f1ac56b2e97873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } health_report { overall_health: UNKNOWN } } }
I20260808 10:34:59.964216 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.960502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.30.146.129:33747)
0808 10:34:59.960657 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.960663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.960679 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:34:59.960790 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.960793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.960858 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.960963 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:59.961257 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.961784 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.962708 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.962943 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.963055 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.963379 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:59.963804 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:59.963932 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.964013 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.964068 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:34:59.964216 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.960347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:33747 (local address 127.0.0.1:44026)
0808 10:34:59.960490 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.960508 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.960645 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.960982 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.960985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.960994 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:59.961116 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.961122 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.961907 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.961911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.962535 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.962543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.963060 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.963063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.963158 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.963203 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:59.963911 (+ 708us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:59.963930 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.964001 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.964093 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 10:34:59.967391 7425 raft_consensus.cc:1261] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Refusing update from remote peer 2f432dfdfa644d4181f1ac56b2e97873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:59.967622 7562 raft_consensus.cc:1261] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Refusing update from remote peer 2f432dfdfa644d4181f1ac56b2e97873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:59.968017 7669 consensus_queue.cc:1035] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:59.968271 7709 consensus_queue.cc:1035] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:59.970150 7668 tablet_replica.cc:439] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: TxnStatusTablet state changed. Reason: New leader 2f432dfdfa644d4181f1ac56b2e97873. Latest consensus state: current_term: 1 leader_uuid: "2f432dfdfa644d4181f1ac56b2e97873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } }
I20260808 10:34:59.970273 7668 tablet_replica.cc:442] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:59.970853 7682 tablet_replica.cc:439] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: TxnStatusTablet state changed. Reason: New leader 2f432dfdfa644d4181f1ac56b2e97873. Latest consensus state: current_term: 1 leader_uuid: "2f432dfdfa644d4181f1ac56b2e97873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } }
I20260808 10:34:59.970975 7682 tablet_replica.cc:442] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:59.972193 7709 tablet_replica.cc:439] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f432dfdfa644d4181f1ac56b2e97873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } }
I20260808 10:34:59.972323 7709 tablet_replica.cc:442] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:34:59.973747 7668 tablet_replica.cc:439] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f432dfdfa644d4181f1ac56b2e97873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } }
I20260808 10:34:59.973855 7668 tablet_replica.cc:442] T 3878cc3ae085443ab1ef028e12951226 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:59.973975 7678 mvcc.cc:205] Tried to move back new op lower bound from 7316214988660703232 to 7316214988409204736. Current Snapshot: MvccSnapshot[applied={T|T < 7316214988660703232 or (T in {7316214988660703232})}]
I20260808 10:34:59.974277 7669 tablet_replica.cc:439] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f432dfdfa644d4181f1ac56b2e97873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } }
I20260808 10:34:59.974377 7669 tablet_replica.cc:442] T 3878cc3ae085443ab1ef028e12951226 P 2f432dfdfa644d4181f1ac56b2e97873: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:34:59.976030 7682 tablet_replica.cc:439] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f432dfdfa644d4181f1ac56b2e97873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } } }
I20260808 10:34:59.976166 7682 tablet_replica.cc:442] T 3878cc3ae085443ab1ef028e12951226 P 28e88b8ed6684731b6275e5e45c5d503: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:34:59.989632 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.985966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:46257 (local address 127.30.146.129:33747)
0808 10:34:59.986098 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:34:59.986102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:34:59.986359 (+ 257us) server_negotiation.cc:407] Connection header received
0808 10:34:59.986527 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:34:59.986530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:34:59.986586 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:34:59.986664 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:34:59.986925 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.987485 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.988362 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.988678 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.988793 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.989080 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:34:59.989221 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:34:59.989304 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:34:59.989429 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 10:34:59.989482 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:34:59.989665 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:34:59.985870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:33747 (local address 127.30.146.131:46257)
0808 10:34:59.986254 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 10:34:59.986277 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 10:34:59.986415 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:34:59.986685 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:34:59.986688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:34:59.986704 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:34:59.986816 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.986821 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.987598 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.987604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.988228 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:34:59.988245 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:34:59.988795 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:34:59.988799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:34:59.988892 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:34:59.988934 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:34:59.989320 (+ 386us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:34:59.989328 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:34:59.989413 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:34:59.989503 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":112,"threads_started":1}
I20260808 10:34:59.997691 7425 raft_consensus.cc:1261] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Refusing update from remote peer 28e88b8ed6684731b6275e5e45c5d503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:59.997787 7262 raft_consensus.cc:1261] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Refusing update from remote peer 28e88b8ed6684731b6275e5e45c5d503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:34:59.998145 7667 consensus_queue.cc:1035] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:34:59.998275 7682 consensus_queue.cc:1035] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:35:00.020177 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:00.015927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:45993 (local address 127.30.146.129:34639)
0808 10:35:00.016303 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:00.016319 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:00.016421 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:00.016819 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:00.016822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:00.016830 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:35:00.016938 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:00.016944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.018012 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.018015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:00.018867 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:00.018877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.019408 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.019412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:00.019497 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:00.019552 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:35:00.019873 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:35:00.019883 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 10:35:00.019949 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:35:00.020032 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":118,"threads_started":1}
I20260808 10:35:00.020169 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:00.016315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:34639 (local address 127.30.146.131:45993)
0808 10:35:00.016427 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:00.016432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:00.016447 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:35:00.016500 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:00.016504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:00.016548 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:00.016804 (+ 256us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:35:00.017072 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.017843 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.019017 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.019286 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.019399 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:00.019692 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:35:00.019784 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:35:00.019866 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:00.019958 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 10:35:00.020016 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:35:00.042230 7081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:35:00.042380 7081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:35:00.044402 7081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb: Sending DeleteTablet for 3 replicas of tablet ea99c162398c44c8b605d99968c5c8be
I20260808 10:35:00.045014 7554 tablet_service.cc:1514] Processing DeleteTablet for tablet ea99c162398c44c8b605d99968c5c8be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:00 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:32846
I20260808 10:35:00.045073 7237 tablet_service.cc:1514] Processing DeleteTablet for tablet ea99c162398c44c8b605d99968c5c8be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:00 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:44014
I20260808 10:35:00.045270 7688 tablet_replica.cc:330] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873: stopping tablet replica
I20260808 10:35:00.045270 7689 tablet_replica.cc:330] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503: stopping tablet replica
I20260808 10:35:00.045405 7688 raft_consensus.cc:2229] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:35:00.045418 7689 raft_consensus.cc:2229] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:35:00.045526 7688 raft_consensus.cc:2258] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:00.045600 7689 raft_consensus.cc:2258] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:00.045790 7398 tablet_service.cc:1514] Processing DeleteTablet for tablet ea99c162398c44c8b605d99968c5c8be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:00 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:35838
I20260808 10:35:00.045974 7687 tablet_replica.cc:330] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: stopping tablet replica
I20260808 10:35:00.046077 7687 raft_consensus.cc:2229] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:35:00.046190 7687 raft_consensus.cc:2258] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:00.046271 7688 ts_tablet_manager.cc:1905] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:35:00.047060 7687 ts_tablet_manager.cc:1905] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:35:00.048621 7688 ts_tablet_manager.cc:1918] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:35:00.048678 7688 log.cc:1196] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/ea99c162398c44c8b605d99968c5c8be
I20260808 10:35:00.048870 7688 ts_tablet_manager.cc:1939] T ea99c162398c44c8b605d99968c5c8be P 2f432dfdfa644d4181f1ac56b2e97873: Deleting consensus metadata
I20260808 10:35:00.049587 7689 ts_tablet_manager.cc:1905] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:35:00.050082 7687 ts_tablet_manager.cc:1918] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:35:00.050174 7687 log.cc:1196] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/ea99c162398c44c8b605d99968c5c8be
I20260808 10:35:00.050490 7687 ts_tablet_manager.cc:1939] T ea99c162398c44c8b605d99968c5c8be P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting consensus metadata
I20260808 10:35:00.051293 7064 catalog_manager.cc:4841] TS 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953): tablet ea99c162398c44c8b605d99968c5c8be (table test-table [id=f67d9f4e26f247f2bac5acd7b43215d1]) successfully deleted
I20260808 10:35:00.051995 7689 ts_tablet_manager.cc:1918] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:35:00.052088 7689 log.cc:1196] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/ea99c162398c44c8b605d99968c5c8be
I20260808 10:35:00.052337 7689 ts_tablet_manager.cc:1939] T ea99c162398c44c8b605d99968c5c8be P 28e88b8ed6684731b6275e5e45c5d503: Deleting consensus metadata
I20260808 10:35:00.053012 7067 catalog_manager.cc:4841] TS 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131:45993): tablet ea99c162398c44c8b605d99968c5c8be (table test-table [id=f67d9f4e26f247f2bac5acd7b43215d1]) successfully deleted
I20260808 10:35:00.056949 7064 catalog_manager.cc:4841] TS 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747): tablet ea99c162398c44c8b605d99968c5c8be (table test-table [id=f67d9f4e26f247f2bac5acd7b43215d1]) successfully deleted
Aug 08 10:35:00 dist-test-slave-9gcw krb5kdc[7033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:35:00 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:00 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:35:00.082568 7139 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:00.075366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.30.146.190:35407)
0808 10:35:00.075521 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:00.075526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:00.075664 (+ 138us) server_negotiation.cc:407] Connection header received
0808 10:35:00.075853 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:00.075855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:00.075904 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:00.075979 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:00.076998 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.077571 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.078762 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.078981 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.079053 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:00.082429 (+ 3376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60720: BlockingRecv error: recv got EOF from 127.0.0.1:60720 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 10:35:00 dist-test-slave-9gcw krb5kdc[7033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM
I20260808 10:35:00.092671 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:00.084385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.30.146.190:35407)
0808 10:35:00.084515 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:00.084519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:00.084614 (+ 95us) server_negotiation.cc:407] Connection header received
0808 10:35:00.084710 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:00.084713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:00.084760 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:00.084879 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:00.085496 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.086000 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.086769 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.086996 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.087101 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:00.089578 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:00.089580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:00.089582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:00.089606 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:00.091016 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:00.091447 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:00.091451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:00.091452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:00.091526 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:00.091806 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:00.091809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:00.091811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:00.091991 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:00.092132 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:00.092380 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 10:35:00.092497 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 10:35:00.095525 7081 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:35:00.096127 7081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:35:00.101043 7237 tablet_service.cc:1467] Processing CreateTablet for tablet 06f6b0834e3349f7ac49d87bdafc41b8 (DEFAULT_TABLE table=test-table [id=5b6b95fcfa5f4dd88a94692a53eb5705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:35:00.101239 7398 tablet_service.cc:1467] Processing CreateTablet for tablet 06f6b0834e3349f7ac49d87bdafc41b8 (DEFAULT_TABLE table=test-table [id=5b6b95fcfa5f4dd88a94692a53eb5705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:35:00.101270 7554 tablet_service.cc:1467] Processing CreateTablet for tablet 06f6b0834e3349f7ac49d87bdafc41b8 (DEFAULT_TABLE table=test-table [id=5b6b95fcfa5f4dd88a94692a53eb5705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:35:00.101401 7237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06f6b0834e3349f7ac49d87bdafc41b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:00.101507 7398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06f6b0834e3349f7ac49d87bdafc41b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:00.101586 7554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06f6b0834e3349f7ac49d87bdafc41b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:00.103677 7659 tablet_bootstrap.cc:492] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503: Bootstrap starting.
I20260808 10:35:00.103866 7662 tablet_bootstrap.cc:492] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873: Bootstrap starting.
I20260808 10:35:00.103866 7663 tablet_bootstrap.cc:492] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Bootstrap starting.
I20260808 10:35:00.104516 7659 tablet_bootstrap.cc:654] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:00.104722 7662 tablet_bootstrap.cc:654] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:00.104736 7663 tablet_bootstrap.cc:654] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:00.105655 7659 tablet_bootstrap.cc:492] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503: No bootstrap required, opened a new log
I20260808 10:35:00.105772 7659 ts_tablet_manager.cc:1397] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:35:00.105962 7663 tablet_bootstrap.cc:492] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: No bootstrap required, opened a new log
I20260808 10:35:00.106071 7663 ts_tablet_manager.cc:1397] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:35:00.106227 7659 raft_consensus.cc:348] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.106312 7662 tablet_bootstrap.cc:492] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873: No bootstrap required, opened a new log
I20260808 10:35:00.106345 7659 raft_consensus.cc:374] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:00.106379 7659 raft_consensus.cc:729] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28e88b8ed6684731b6275e5e45c5d503, State: Initialized, Role: FOLLOWER
I20260808 10:35:00.106386 7662 ts_tablet_manager.cc:1397] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 10:35:00.106508 7659 consensus_queue.cc:260] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.106639 7663 raft_consensus.cc:348] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.106755 7663 raft_consensus.cc:374] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:00.106781 7659 ts_tablet_manager.cc:1428] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:35:00.106788 7663 raft_consensus.cc:729] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbb9e566f3d4b22bcdcfc1e2b21d5ae, State: Initialized, Role: FOLLOWER
I20260808 10:35:00.106770 7662 raft_consensus.cc:348] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.106884 7662 raft_consensus.cc:374] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:00.106917 7662 raft_consensus.cc:729] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f432dfdfa644d4181f1ac56b2e97873, State: Initialized, Role: FOLLOWER
I20260808 10:35:00.106913 7663 consensus_queue.cc:260] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.107030 7662 consensus_queue.cc:260] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.107312 7662 ts_tablet_manager.cc:1428] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:35:00.107920 7663 ts_tablet_manager.cc:1428] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:35:00.198626 7667 raft_consensus.cc:482] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:35:00.198801 7667 raft_consensus.cc:504] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.199369 7667 leader_election.cc:288] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747), 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953)
I20260808 10:35:00.199905 7262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f6b0834e3349f7ac49d87bdafc41b8" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f432dfdfa644d4181f1ac56b2e97873" is_pre_election: true
I20260808 10:35:00.199985 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f6b0834e3349f7ac49d87bdafc41b8" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" is_pre_election: true
I20260808 10:35:00.200101 7262 raft_consensus.cc:2454] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 0.
I20260808 10:35:00.200181 7425 raft_consensus.cc:2454] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 0.
I20260808 10:35:00.200476 7498 leader_election.cc:302] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e88b8ed6684731b6275e5e45c5d503, 2f432dfdfa644d4181f1ac56b2e97873; no voters:
I20260808 10:35:00.200718 7667 raft_consensus.cc:2781] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:35:00.200789 7667 raft_consensus.cc:482] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:35:00.200834 7667 raft_consensus.cc:3037] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:00.201998 7667 raft_consensus.cc:504] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.202409 7667 leader_election.cc:288] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 election: Requested vote from peers 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747), 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953)
I20260808 10:35:00.202813 7262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f6b0834e3349f7ac49d87bdafc41b8" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f432dfdfa644d4181f1ac56b2e97873"
I20260808 10:35:00.202876 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f6b0834e3349f7ac49d87bdafc41b8" candidate_uuid: "28e88b8ed6684731b6275e5e45c5d503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae"
I20260808 10:35:00.202958 7262 raft_consensus.cc:3037] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:00.202993 7425 raft_consensus.cc:3037] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:00.204200 7425 raft_consensus.cc:2454] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 1.
I20260808 10:35:00.204200 7262 raft_consensus.cc:2454] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28e88b8ed6684731b6275e5e45c5d503 in term 1.
I20260808 10:35:00.204525 7498 leader_election.cc:302] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e88b8ed6684731b6275e5e45c5d503, 2f432dfdfa644d4181f1ac56b2e97873; no voters:
I20260808 10:35:00.204780 7667 raft_consensus.cc:2781] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:35:00.204918 7667 raft_consensus.cc:686] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 LEADER]: Becoming Leader. State: Replica: 28e88b8ed6684731b6275e5e45c5d503, State: Running, Role: LEADER
I20260808 10:35:00.205081 7667 consensus_queue.cc:237] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.206899 7082 catalog_manager.cc:5495] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131). New cstate: current_term: 1 leader_uuid: "28e88b8ed6684731b6275e5e45c5d503" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } health_report { overall_health: HEALTHY } } }
I20260808 10:35:00.232699 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:00.229234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.30.146.131:45993)
0808 10:35:00.229380 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:00.229385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:00.229404 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:35:00.229560 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:00.229563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:00.229645 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:00.229751 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:35:00.230063 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.230632 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.231542 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.231774 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.231900 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:00.232202 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:35:00.232278 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:35:00.232367 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:00.232468 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 10:35:00.232526 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:35:00.235173 7262 raft_consensus.cc:1261] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Refusing update from remote peer 28e88b8ed6684731b6275e5e45c5d503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:35:00.235159 7425 raft_consensus.cc:1261] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Refusing update from remote peer 28e88b8ed6684731b6275e5e45c5d503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:35:00.235692 7667 consensus_queue.cc:1035] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:35:00.235894 7671 consensus_queue.cc:1035] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:35:00.243114 7082 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:35:00.243289 7082 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:35:00.245496 7082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb: Sending DeleteTablet for 3 replicas of tablet 06f6b0834e3349f7ac49d87bdafc41b8
I20260808 10:35:00.246037 7237 tablet_service.cc:1514] Processing DeleteTablet for tablet 06f6b0834e3349f7ac49d87bdafc41b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:00 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:44014
I20260808 10:35:00.246136 7554 tablet_service.cc:1514] Processing DeleteTablet for tablet 06f6b0834e3349f7ac49d87bdafc41b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:00 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:32846
I20260808 10:35:00.246304 7689 tablet_replica.cc:330] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503: stopping tablet replica
I20260808 10:35:00.246284 7398 tablet_service.cc:1514] Processing DeleteTablet for tablet 06f6b0834e3349f7ac49d87bdafc41b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:00 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:35838
I20260808 10:35:00.246438 7689 raft_consensus.cc:2229] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:35:00.246523 7688 tablet_replica.cc:330] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873: stopping tablet replica
I20260808 10:35:00.246639 7689 raft_consensus.cc:2258] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:00.246784 7688 raft_consensus.cc:2229] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:35:00.246963 7688 raft_consensus.cc:2258] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:00.247174 7687 tablet_replica.cc:330] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: stopping tablet replica
I20260808 10:35:00.247157 7082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:35:00.247319 7687 raft_consensus.cc:2229] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:35:00.247488 7687 raft_consensus.cc:2258] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:00.247524 7689 ts_tablet_manager.cc:1905] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:35:00.247648 7082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:35:00.248286 7687 ts_tablet_manager.cc:1905] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:35:00.248286 7688 ts_tablet_manager.cc:1905] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:35:00.249799 7689 ts_tablet_manager.cc:1918] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:35:00.249894 7689 log.cc:1196] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/06f6b0834e3349f7ac49d87bdafc41b8
I20260808 10:35:00.250149 7689 ts_tablet_manager.cc:1939] T 06f6b0834e3349f7ac49d87bdafc41b8 P 28e88b8ed6684731b6275e5e45c5d503: Deleting consensus metadata
I20260808 10:35:00.250857 7067 catalog_manager.cc:4841] TS 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131:45993): tablet 06f6b0834e3349f7ac49d87bdafc41b8 (table test-table [id=5b6b95fcfa5f4dd88a94692a53eb5705]) successfully deleted
I20260808 10:35:00.250947 7688 ts_tablet_manager.cc:1918] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:35:00.250947 7687 ts_tablet_manager.cc:1918] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:35:00.251037 7688 log.cc:1196] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/06f6b0834e3349f7ac49d87bdafc41b8
I20260808 10:35:00.251037 7687 log.cc:1196] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/06f6b0834e3349f7ac49d87bdafc41b8
I20260808 10:35:00.251351 7688 ts_tablet_manager.cc:1939] T 06f6b0834e3349f7ac49d87bdafc41b8 P 2f432dfdfa644d4181f1ac56b2e97873: Deleting consensus metadata
I20260808 10:35:00.251351 7687 ts_tablet_manager.cc:1939] T 06f6b0834e3349f7ac49d87bdafc41b8 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting consensus metadata
I20260808 10:35:00.252108 7064 catalog_manager.cc:4841] TS 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953): tablet 06f6b0834e3349f7ac49d87bdafc41b8 (table test-table [id=5b6b95fcfa5f4dd88a94692a53eb5705]) successfully deleted
I20260808 10:35:00.252295 7064 catalog_manager.cc:4841] TS 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747): tablet 06f6b0834e3349f7ac49d87bdafc41b8 (table test-table [id=5b6b95fcfa5f4dd88a94692a53eb5705]) successfully deleted
I20260808 10:35:00.256076 7237 tablet_service.cc:1467] Processing CreateTablet for tablet b9e4016ee46b49af9fa1a33e7bc64db5 (DEFAULT_TABLE table=test-table [id=4dbb94bb4a3c460f94dcd8962dbbca91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:35:00.256134 7398 tablet_service.cc:1467] Processing CreateTablet for tablet b9e4016ee46b49af9fa1a33e7bc64db5 (DEFAULT_TABLE table=test-table [id=4dbb94bb4a3c460f94dcd8962dbbca91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:35:00.256377 7398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9e4016ee46b49af9fa1a33e7bc64db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:00.256377 7237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9e4016ee46b49af9fa1a33e7bc64db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:00.256379 7554 tablet_service.cc:1467] Processing CreateTablet for tablet b9e4016ee46b49af9fa1a33e7bc64db5 (DEFAULT_TABLE table=test-table [id=4dbb94bb4a3c460f94dcd8962dbbca91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:35:00.256652 7554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9e4016ee46b49af9fa1a33e7bc64db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:00.259047 7663 tablet_bootstrap.cc:492] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Bootstrap starting.
I20260808 10:35:00.259048 7659 tablet_bootstrap.cc:492] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503: Bootstrap starting.
I20260808 10:35:00.259943 7663 tablet_bootstrap.cc:654] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:00.259964 7659 tablet_bootstrap.cc:654] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:00.260676 7662 tablet_bootstrap.cc:492] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873: Bootstrap starting.
I20260808 10:35:00.261251 7663 tablet_bootstrap.cc:492] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: No bootstrap required, opened a new log
I20260808 10:35:00.261340 7663 ts_tablet_manager.cc:1397] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:35:00.261653 7662 tablet_bootstrap.cc:654] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:00.261929 7659 tablet_bootstrap.cc:492] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503: No bootstrap required, opened a new log
I20260808 10:35:00.261915 7663 raft_consensus.cc:348] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.262040 7663 raft_consensus.cc:374] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:00.262051 7659 ts_tablet_manager.cc:1397] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 10:35:00.262091 7663 raft_consensus.cc:729] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bbb9e566f3d4b22bcdcfc1e2b21d5ae, State: Initialized, Role: FOLLOWER
I20260808 10:35:00.262262 7663 consensus_queue.cc:260] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.262620 7663 ts_tablet_manager.cc:1428] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:35:00.262580 7659 raft_consensus.cc:348] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.262753 7659 raft_consensus.cc:374] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:00.262801 7659 raft_consensus.cc:729] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28e88b8ed6684731b6275e5e45c5d503, State: Initialized, Role: FOLLOWER
I20260808 10:35:00.262950 7659 consensus_queue.cc:260] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.263306 7659 ts_tablet_manager.cc:1428] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:35:00.263487 7662 tablet_bootstrap.cc:492] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873: No bootstrap required, opened a new log
I20260808 10:35:00.263576 7662 ts_tablet_manager.cc:1397] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 10:35:00.264029 7662 raft_consensus.cc:348] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.264189 7662 raft_consensus.cc:374] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:00.264249 7662 raft_consensus.cc:729] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f432dfdfa644d4181f1ac56b2e97873, State: Initialized, Role: FOLLOWER
I20260808 10:35:00.264400 7662 consensus_queue.cc:260] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.264757 7662 ts_tablet_manager.cc:1428] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:35:00.509778 7669 raft_consensus.cc:482] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:35:00.509935 7669 raft_consensus.cc:504] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.510367 7669 leader_election.cc:288] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953), 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131:45993)
I20260808 10:35:00.510910 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e4016ee46b49af9fa1a33e7bc64db5" candidate_uuid: "2f432dfdfa644d4181f1ac56b2e97873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28e88b8ed6684731b6275e5e45c5d503" is_pre_election: true
I20260808 10:35:00.511062 7562 raft_consensus.cc:2454] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f432dfdfa644d4181f1ac56b2e97873 in term 0.
I20260808 10:35:00.511121 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e4016ee46b49af9fa1a33e7bc64db5" candidate_uuid: "2f432dfdfa644d4181f1ac56b2e97873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" is_pre_election: true
I20260808 10:35:00.511351 7425 raft_consensus.cc:2454] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f432dfdfa644d4181f1ac56b2e97873 in term 0.
I20260808 10:35:00.511457 7187 leader_election.cc:302] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e88b8ed6684731b6275e5e45c5d503, 2f432dfdfa644d4181f1ac56b2e97873; no voters:
I20260808 10:35:00.511704 7669 raft_consensus.cc:2781] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:35:00.511798 7669 raft_consensus.cc:482] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:35:00.511852 7669 raft_consensus.cc:3037] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:00.512971 7669 raft_consensus.cc:504] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.513398 7669 leader_election.cc:288] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [CANDIDATE]: Term 1 election: Requested vote from peers 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953), 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131:45993)
I20260808 10:35:00.513898 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e4016ee46b49af9fa1a33e7bc64db5" candidate_uuid: "2f432dfdfa644d4181f1ac56b2e97873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae"
I20260808 10:35:00.513953 7562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e4016ee46b49af9fa1a33e7bc64db5" candidate_uuid: "2f432dfdfa644d4181f1ac56b2e97873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28e88b8ed6684731b6275e5e45c5d503"
I20260808 10:35:00.514027 7425 raft_consensus.cc:3037] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:00.514133 7562 raft_consensus.cc:3037] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:00.515129 7562 raft_consensus.cc:2454] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f432dfdfa644d4181f1ac56b2e97873 in term 1.
I20260808 10:35:00.515144 7425 raft_consensus.cc:2454] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f432dfdfa644d4181f1ac56b2e97873 in term 1.
I20260808 10:35:00.515488 7187 leader_election.cc:302] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e88b8ed6684731b6275e5e45c5d503, 2f432dfdfa644d4181f1ac56b2e97873; no voters:
I20260808 10:35:00.515754 7669 raft_consensus.cc:2781] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:35:00.515879 7669 raft_consensus.cc:686] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 LEADER]: Becoming Leader. State: Replica: 2f432dfdfa644d4181f1ac56b2e97873, State: Running, Role: LEADER
I20260808 10:35:00.516054 7669 consensus_queue.cc:237] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } }
I20260808 10:35:00.517975 7082 catalog_manager.cc:5495] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129). New cstate: current_term: 1 leader_uuid: "2f432dfdfa644d4181f1ac56b2e97873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f432dfdfa644d4181f1ac56b2e97873" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 } health_report { overall_health: UNKNOWN } } }
I20260808 10:35:00.572389 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:00.568668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.30.146.129:33747)
0808 10:35:00.569305 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:00.569311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:00.569326 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:35:00.569375 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:00.569379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:00.569434 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:00.569525 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:35:00.569785 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.570332 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.571271 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.571520 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.571633 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:00.571893 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:35:00.571978 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:35:00.572059 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:00.572157 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 10:35:00.572212 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":82,"threads_started":1}
I20260808 10:35:00.582373 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:00.579162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:59069 (local address 127.30.146.129:33747)
0808 10:35:00.579308 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:00.579311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:00.579359 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:35:00.579480 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:00.579482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:00.579520 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:00.579574 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:35:00.579883 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.580368 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.581369 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.581574 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.581650 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:00.581970 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:35:00.582036 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:35:00.582114 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:00.582199 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 10:35:00.582242 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:35:00.582398 7760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:00.578965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:33747 (local address 127.30.146.129:59069)
0808 10:35:00.579273 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:00.579290 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:00.579398 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:00.579603 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:00.579605 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:00.579612 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:35:00.579801 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:00.579806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.580489 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.580491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:00.581268 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:00.581280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:00.581721 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:00.581724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:00.581792 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:00.581820 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:35:00.582126 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:35:00.582133 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:35:00.582191 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:35:00.582262 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":169,"threads_started":1}
I20260808 10:35:00.588622 7562 raft_consensus.cc:1261] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Refusing update from remote peer 2f432dfdfa644d4181f1ac56b2e97873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:35:00.588641 7425 raft_consensus.cc:1261] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Refusing update from remote peer 2f432dfdfa644d4181f1ac56b2e97873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:35:00.589018 7756 consensus_queue.cc:1035] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bbb9e566f3d4b22bcdcfc1e2b21d5ae" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:35:00.589223 7720 consensus_queue.cc:1035] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28e88b8ed6684731b6275e5e45c5d503" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:35:00.628011 7081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:35:00.628217 7081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:35:00.630702 7081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6faa9f501ce5474cbdec7d05706e88eb: Sending DeleteTablet for 3 replicas of tablet b9e4016ee46b49af9fa1a33e7bc64db5
I20260808 10:35:00.631654 7554 tablet_service.cc:1514] Processing DeleteTablet for tablet b9e4016ee46b49af9fa1a33e7bc64db5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:00 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:32846
I20260808 10:35:00.631685 7398 tablet_service.cc:1514] Processing DeleteTablet for tablet b9e4016ee46b49af9fa1a33e7bc64db5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:00 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:35838
I20260808 10:35:00.631873 7689 tablet_replica.cc:330] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503: stopping tablet replica
I20260808 10:35:00.631955 7687 tablet_replica.cc:330] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: stopping tablet replica
I20260808 10:35:00.632021 7689 raft_consensus.cc:2229] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:35:00.632084 7687 raft_consensus.cc:2229] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:35:00.632161 7689 raft_consensus.cc:2258] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:00.632265 7687 raft_consensus.cc:2258] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:00.632388 7237 tablet_service.cc:1514] Processing DeleteTablet for tablet b9e4016ee46b49af9fa1a33e7bc64db5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:00 UTC) from {username='oryx', principal='oryx/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:44014
I20260808 10:35:00.632602 7688 tablet_replica.cc:330] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873: stopping tablet replica
I20260808 10:35:00.632733 7688 raft_consensus.cc:2229] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:35:00.633126 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 7164
I20260808 10:35:00.633119 7688 raft_consensus.cc:2258] T b9e4016ee46b49af9fa1a33e7bc64db5 P 2f432dfdfa644d4181f1ac56b2e97873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:00.633962 7687 ts_tablet_manager.cc:1905] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:35:00.634446 7689 ts_tablet_manager.cc:1905] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:35:00.636861 7689 ts_tablet_manager.cc:1918] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:35:00.636973 7689 log.cc:1196] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/b9e4016ee46b49af9fa1a33e7bc64db5
I20260808 10:35:00.637338 7689 ts_tablet_manager.cc:1939] T b9e4016ee46b49af9fa1a33e7bc64db5 P 28e88b8ed6684731b6275e5e45c5d503: Deleting consensus metadata
I20260808 10:35:00.637792 7687 ts_tablet_manager.cc:1918] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:35:00.637868 7687 log.cc:1196] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/b9e4016ee46b49af9fa1a33e7bc64db5
I20260808 10:35:00.638214 7067 catalog_manager.cc:4841] TS 28e88b8ed6684731b6275e5e45c5d503 (127.30.146.131:45993): tablet b9e4016ee46b49af9fa1a33e7bc64db5 (table test-table [id=4dbb94bb4a3c460f94dcd8962dbbca91]) successfully deleted
I20260808 10:35:00.638474 7687 ts_tablet_manager.cc:1939] T b9e4016ee46b49af9fa1a33e7bc64db5 P 6bbb9e566f3d4b22bcdcfc1e2b21d5ae: Deleting consensus metadata
I20260808 10:35:00.639222 7064 catalog_manager.cc:4841] TS 6bbb9e566f3d4b22bcdcfc1e2b21d5ae (127.30.146.130:41953): tablet b9e4016ee46b49af9fa1a33e7bc64db5 (table test-table [id=4dbb94bb4a3c460f94dcd8962dbbca91]) successfully deleted
W20260808 10:35:00.647580 7064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:33747 (error 108)
I20260808 10:35:00.647768 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 7323
I20260808 10:35:00.651621 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:00.651125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:33747 (local address 127.0.0.1:44050)
0808 10:35:00.651442 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:00.651520 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:33747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":148,"threads_started":1}
W20260808 10:35:00.651773 7064 catalog_manager.cc:4584] TS 2f432dfdfa644d4181f1ac56b2e97873 (127.30.146.129:33747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9e4016ee46b49af9fa1a33e7bc64db5: Network error: Client connection negotiation failed: client connection to 127.30.146.129:33747: connect: Connection refused (error 111)
I20260808 10:35:00.657019 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 7482
I20260808 10:35:00.666318 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 7048
2026-08-08T10:35:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3716 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:35:00 dist-test-slave-9gcw krb5kdc[7771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:35:00 dist-test-slave-9gcw krb5kdc[7771](info): set up 2 sockets
Aug 08 10:35:00 dist-test-slave-9gcw krb5kdc[7771](info): commencing operation
krb5kdc: starting...
W20260808 10:35:01.850019 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:35:01 dist-test-slave-9gcw krb5kdc[7771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:35:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:35:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:35:01.993496 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:44621
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:44621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:35:02.113065 7786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:02.113384 7786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:02.113426 7786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:02.117803 7786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:35:02.117887 7786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:02.117908 7786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:02.117926 7786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:35:02.117942 7786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:35:02.123546 7786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:44621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:44621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:02.124691 7786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:02.125870 7786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:02.135550 7794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:02.135620 7791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:02.135643 7786 server_base.cc:1034] running on GCE node
W20260808 10:35:02.135905 7792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:02.136477 7786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:02.137810 7786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:02.139030 7786 hybrid_clock.cc:648] HybridClock initialized: now 1786185302138999 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T10:35:02Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1483 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:35:02 dist-test-slave-9gcw krb5kdc[7799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:35:02 dist-test-slave-9gcw krb5kdc[7799](info): set up 2 sockets
Aug 08 10:35:02 dist-test-slave-9gcw krb5kdc[7799](info): commencing operation
krb5kdc: starting...
W20260808 10:35:03.252832 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:35:03 dist-test-slave-9gcw krb5kdc[7799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:35:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:35:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:35:03.394588 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:34763
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:34763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:35:03.511391 7812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:03.511745 7812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:03.511840 7812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:03.515959 7812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:35:03.516065 7812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:03.516105 7812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:03.516145 7812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:35:03.516180 7812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:35:03.521891 7812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:34763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:34763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:03.523066 7812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:03.524166 7812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:03.533365 7818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:03.533417 7820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:03.533365 7817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:03.533771 7812 server_base.cc:1034] running on GCE node
I20260808 10:35:03.534447 7812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:03.535723 7812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:03.536927 7812 hybrid_clock.cc:648] HybridClock initialized: now 1786185303536917 us; error 28 us; skew 500 ppm
Aug 08 10:35:03 dist-test-slave-9gcw krb5kdc[7799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185303, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:03.539958 7812 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:35:03.540923 7812 webserver.cc:460] Webserver started at http://127.30.146.190:42915/ using document root <none> and password file <none>
I20260808 10:35:03.541558 7812 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:03.541653 7812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:03.541903 7812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:03.544030 7812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "5b71f6a8dedc4ecb8672f95101055650"
format_stamp: "Formatted at 2026-08-08 10:35:03 on dist-test-slave-9gcw"
server_key: "f750a231ca924a9ca0d50a2f24f4b51b"
server_key_iv: "721b218f695f06adb033ee573b4339ff"
server_key_version: "encryptionkey@0"
I20260808 10:35:03.544608 7812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "5b71f6a8dedc4ecb8672f95101055650"
format_stamp: "Formatted at 2026-08-08 10:35:03 on dist-test-slave-9gcw"
server_key: "f750a231ca924a9ca0d50a2f24f4b51b"
server_key_iv: "721b218f695f06adb033ee573b4339ff"
server_key_version: "encryptionkey@0"
I20260808 10:35:03.548399 7812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:35:03.551056 7831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:03.552521 7812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:35:03.552625 7812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "5b71f6a8dedc4ecb8672f95101055650"
format_stamp: "Formatted at 2026-08-08 10:35:03 on dist-test-slave-9gcw"
server_key: "f750a231ca924a9ca0d50a2f24f4b51b"
server_key_iv: "721b218f695f06adb033ee573b4339ff"
server_key_version: "encryptionkey@0"
I20260808 10:35:03.552706 7812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:03.572540 7812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:03.575785 7812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:03.575947 7812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:03.583962 7812 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:34763
I20260808 10:35:03.584038 7889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:34763 every 8 connection(s)
I20260808 10:35:03.585124 7812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:35:03.588167 7891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:03.593474 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 7812
I20260808 10:35:03.593626 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:35:03.593969 31306 external_mini_cluster.cc:1428] Setting key dd7a881be0b860b68aff20050ede9f31
I20260808 10:35:03.594198 7891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: Bootstrap starting.
I20260808 10:35:03.596908 7891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:03.597761 7891 log.cc:824] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: Log is configured to *not* fsync() on all Append() calls
I20260808 10:35:03.600107 7891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: No bootstrap required, opened a new log
Aug 08 10:35:03 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:35:03.604619 7891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } }
I20260808 10:35:03.604919 7891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:03.604997 7891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b71f6a8dedc4ecb8672f95101055650, State: Initialized, Role: FOLLOWER
I20260808 10:35:03.605566 7891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } }
I20260808 10:35:03.605756 7891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:35:03.605829 7891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:35:03.605945 7891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:03.607260 7891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } }
I20260808 10:35:03.607636 7891 leader_election.cc:302] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b71f6a8dedc4ecb8672f95101055650; no voters:
I20260808 10:35:03.608059 7891 leader_election.cc:288] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:35:03.608180 7896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:35:03.608446 7896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 1 LEADER]: Becoming Leader. State: Replica: 5b71f6a8dedc4ecb8672f95101055650, State: Running, Role: LEADER
I20260808 10:35:03.608808 7896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } }
I20260808 10:35:03.609537 7891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:35:03.611023 7894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:03.595470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.30.146.190:34763)
0808 10:35:03.595969 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:03.595984 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:03.596020 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:35:03.596666 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:03.596682 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:03.597059 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:03.597515 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:03.598498 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:03.599995 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:03.600938 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:03.601654 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:03.601818 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:03.604881 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:03.604888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:03.604906 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:03.604942 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:03.607272 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:03.607778 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:03.607784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:03.607790 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:03.607942 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:03.608334 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:03.608337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:03.608339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:03.608983 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:03.609201 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:03.609575 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 10:35:03.609915 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":141,"threads_started":1}
I20260808 10:35:03.611040 7898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b71f6a8dedc4ecb8672f95101055650. Latest consensus state: current_term: 1 leader_uuid: "5b71f6a8dedc4ecb8672f95101055650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } } }
I20260808 10:35:03.611090 7897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b71f6a8dedc4ecb8672f95101055650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } } }
I20260808 10:35:03.611238 7897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [sys.catalog]: This master's current role is: LEADER
I20260808 10:35:03.611243 7898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [sys.catalog]: This master's current role is: LEADER
W20260808 10:35:03.615068 7914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:35:03.615156 7914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:35:03.615433 7912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:35:03.616328 7912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:35:03.622416 7912 catalog_manager.cc:1347] Generated new cluster ID: 5ed84649485948298d320779457f40e2
I20260808 10:35:03.622495 7912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:35:03.633229 7912 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:35:03.634322 7912 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:35:03.644107 7912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: Generated new TSK 0
I20260808 10:35:03.645026 7912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:35:03.680485 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34763
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:35:03.798578 7922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:03.798934 7922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:03.799039 7922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:03.803174 7922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:03.803280 7922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:03.803380 7922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:35:03.808709 7922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:03.810065 7922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:03.811321 7922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:03.821475 7928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:03.821467 7930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:03.821800 7922 server_base.cc:1034] running on GCE node
W20260808 10:35:03.821497 7927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:03.822360 7922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:03.823014 7922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:03.824330 7922 hybrid_clock.cc:648] HybridClock initialized: now 1786185303824284 us; error 75 us; skew 500 ppm
Aug 08 10:35:03 dist-test-slave-9gcw krb5kdc[7799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185303, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:03.827765 7922 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:35:03.828779 7922 webserver.cc:460] Webserver started at http://127.30.146.129:40515/ using document root <none> and password file <none>
I20260808 10:35:03.829525 7922 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:03.829597 7922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:03.829785 7922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:03.832070 7922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "9ae6458bbdce4d4097ff597dac075295"
format_stamp: "Formatted at 2026-08-08 10:35:03 on dist-test-slave-9gcw"
server_key: "a646e6e7d1b070aae97f61edc03f9e7f"
server_key_iv: "e94e95699bb5c886ed8a39cc43a6583f"
server_key_version: "encryptionkey@0"
I20260808 10:35:03.832963 7922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "9ae6458bbdce4d4097ff597dac075295"
format_stamp: "Formatted at 2026-08-08 10:35:03 on dist-test-slave-9gcw"
server_key: "a646e6e7d1b070aae97f61edc03f9e7f"
server_key_iv: "e94e95699bb5c886ed8a39cc43a6583f"
server_key_version: "encryptionkey@0"
I20260808 10:35:03.837728 7922 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:35:03.840363 7937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:03.841970 7922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:35:03.842150 7922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "9ae6458bbdce4d4097ff597dac075295"
format_stamp: "Formatted at 2026-08-08 10:35:03 on dist-test-slave-9gcw"
server_key: "a646e6e7d1b070aae97f61edc03f9e7f"
server_key_iv: "e94e95699bb5c886ed8a39cc43a6583f"
server_key_version: "encryptionkey@0"
I20260808 10:35:03.842326 7922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:03.870932 7922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:03.874534 7922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:03.874771 7922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:03.875474 7922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:03.876432 7922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:03.876505 7922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:03.876582 7922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:03.876633 7922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:03.886545 7922 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:44991
I20260808 10:35:03.886572 8059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:44991 every 8 connection(s)
I20260808 10:35:03.887501 7922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:35:03.888705 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 7922
I20260808 10:35:03.888862 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:35:03.889245 31306 external_mini_cluster.cc:1428] Setting key 8c6ccccdfb9a5a80c3554bc7ea15b455
Aug 08 10:35:03 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185303, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:35:03.904678 7894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:03.889544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:54345 (local address 127.30.146.190:34763)
0808 10:35:03.889772 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:03.889779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:03.890667 (+ 888us) server_negotiation.cc:407] Connection header received
0808 10:35:03.892116 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:03.892120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:03.892174 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:03.892270 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:03.894014 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:03.894633 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:03.896569 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:03.896809 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:03.896891 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:03.900248 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:03.900252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:03.900254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:03.900284 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:03.902236 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:03.902744 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:03.902747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:03.902748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:03.902889 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:03.903266 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:03.903269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:03.903270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:03.903576 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:03.903681 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:03.904340 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 10:35:03.904488 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260808 10:35:03.905640 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:03.889977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:54345)
0808 10:35:03.890508 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:03.890564 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:03.891887 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:03.892445 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:03.892452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:03.893009 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:03.893826 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:03.893850 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:03.894798 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:03.894804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:03.896422 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:03.896436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:03.896997 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:03.897003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:03.897299 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:03.898006 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:03.898034 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:03.900001 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:03.902360 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:03.902365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:03.902380 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:03.902645 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:03.903023 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:03.903026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:03.903027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:03.903172 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:03.903698 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:03.903708 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:03.904064 (+ 356us) client_negotiation.cc:769] Sending connection context
0808 10:35:03.904270 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 10:35:03.904621 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":155,"threads_started":1}
I20260808 10:35:03.906983 8060 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34763
I20260808 10:35:03.907326 8060 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:03.907970 8060 heartbeater.cc:511] Master 127.30.146.190:34763 requested a full tablet report, sending...
I20260808 10:35:03.909930 7850 ts_manager.cc:194] Registered new tserver with Master: 9ae6458bbdce4d4097ff597dac075295 (127.30.146.129:44991)
I20260808 10:35:03.911393 7850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:54345
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:35:03.925266 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34763
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:35:04.043203 8065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:04.043469 8065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:04.043514 8065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:04.047683 8065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:04.047745 8065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:04.047804 8065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:35:04.053045 8065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:04.054180 8065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:04.055178 8065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:04.065237 8072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:04.065263 8074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:04.065361 8065 server_base.cc:1034] running on GCE node
W20260808 10:35:04.065264 8071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:04.065896 8065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:04.066533 8065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:04.067720 8065 hybrid_clock.cc:648] HybridClock initialized: now 1786185304067703 us; error 29 us; skew 500 ppm
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:04.071130 8065 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:35:04.072088 8065 webserver.cc:460] Webserver started at http://127.30.146.130:43417/ using document root <none> and password file <none>
I20260808 10:35:04.072669 8065 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:04.072780 8065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:04.073015 8065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:04.075145 8065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "386d19185ccd4204b394130bb4e28a62"
format_stamp: "Formatted at 2026-08-08 10:35:04 on dist-test-slave-9gcw"
server_key: "cfa609fafeca45665ab2185408b71f67"
server_key_iv: "dee32b1f0d86ac29e6304b7ee60a625b"
server_key_version: "encryptionkey@0"
I20260808 10:35:04.075758 8065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "386d19185ccd4204b394130bb4e28a62"
format_stamp: "Formatted at 2026-08-08 10:35:04 on dist-test-slave-9gcw"
server_key: "cfa609fafeca45665ab2185408b71f67"
server_key_iv: "dee32b1f0d86ac29e6304b7ee60a625b"
server_key_version: "encryptionkey@0"
I20260808 10:35:04.079643 8065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:35:04.082295 8081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:04.083686 8065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:35:04.083848 8065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "386d19185ccd4204b394130bb4e28a62"
format_stamp: "Formatted at 2026-08-08 10:35:04 on dist-test-slave-9gcw"
server_key: "cfa609fafeca45665ab2185408b71f67"
server_key_iv: "dee32b1f0d86ac29e6304b7ee60a625b"
server_key_version: "encryptionkey@0"
I20260808 10:35:04.083971 8065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:04.094317 8065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:04.097756 8065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:04.097995 8065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:04.098667 8065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:04.099596 8065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:04.099656 8065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:04.099701 8065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:04.099720 8065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:04.109936 8065 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:33557
I20260808 10:35:04.109964 8213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:33557 every 8 connection(s)
I20260808 10:35:04.111174 8065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:35:04.113286 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 8065
I20260808 10:35:04.113427 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:35:04.113796 31306 external_mini_cluster.cc:1428] Setting key e58c23d0d4e06f4c7098327e229d354d
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:35:04.128036 7894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:04.113382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:52203 (local address 127.30.146.190:34763)
0808 10:35:04.113555 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:04.113561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:04.115104 (+ 1543us) server_negotiation.cc:407] Connection header received
0808 10:35:04.116284 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:04.116290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:04.116356 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:04.116468 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:04.117929 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.118528 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.119838 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.120278 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.120442 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:04.123894 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:04.123903 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:04.123905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:04.123934 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:04.125655 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:04.126274 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:04.126277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:04.126279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:04.126361 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:04.126817 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:04.126819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:04.126821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:04.127017 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:04.127135 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:04.127707 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 10:35:04.127862 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
I20260808 10:35:04.129067 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:04.113833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:52203)
0808 10:35:04.114905 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:04.114966 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:04.116041 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:04.116647 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:04.116657 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:04.117076 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:04.117759 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:04.117778 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.118684 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.118688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:04.119682 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:04.119693 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.120484 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.120489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:04.120693 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:04.121478 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:04.121497 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:04.123618 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:04.125817 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:04.125823 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:04.125840 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:04.126134 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:04.126466 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:04.126469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:04.126471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:04.126631 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:04.127139 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:04.127149 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:04.127436 (+ 287us) client_negotiation.cc:769] Sending connection context
0808 10:35:04.127635 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 10:35:04.128093 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":603,"threads_started":1}
I20260808 10:35:04.130432 8214 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34763
I20260808 10:35:04.130692 8214 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:04.131453 8214 heartbeater.cc:511] Master 127.30.146.190:34763 requested a full tablet report, sending...
I20260808 10:35:04.132664 7850 ts_manager.cc:194] Registered new tserver with Master: 386d19185ccd4204b394130bb4e28a62 (127.30.146.130:33557)
I20260808 10:35:04.133359 7850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:52203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:35:04.148928 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34763
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:35:04.264057 8219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:04.264375 8219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:04.264456 8219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:04.269196 8219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:04.269325 8219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:04.269425 8219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:35:04.275020 8219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:04.276204 8219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:04.277292 8219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:04.287572 8226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:04.287696 8228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:04.287596 8225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:04.287796 8219 server_base.cc:1034] running on GCE node
I20260808 10:35:04.288337 8219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:04.288985 8219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:04.290161 8219 hybrid_clock.cc:648] HybridClock initialized: now 1786185304290144 us; error 29 us; skew 500 ppm
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:04.293064 8219 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:35:04.294090 8219 webserver.cc:460] Webserver started at http://127.30.146.131:46329/ using document root <none> and password file <none>
I20260808 10:35:04.294612 8219 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:04.294673 8219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:04.294855 8219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:04.297381 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "d232722170904b12a049bf0142550ace"
format_stamp: "Formatted at 2026-08-08 10:35:04 on dist-test-slave-9gcw"
server_key: "a92e153fc9ce007fcba9ffab477ced59"
server_key_iv: "35fab7e5652d583eee540690b521c8d9"
server_key_version: "encryptionkey@0"
I20260808 10:35:04.298069 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "d232722170904b12a049bf0142550ace"
format_stamp: "Formatted at 2026-08-08 10:35:04 on dist-test-slave-9gcw"
server_key: "a92e153fc9ce007fcba9ffab477ced59"
server_key_iv: "35fab7e5652d583eee540690b521c8d9"
server_key_version: "encryptionkey@0"
I20260808 10:35:04.302182 8219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:35:04.304746 8236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:04.306201 8219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:35:04.306322 8219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "d232722170904b12a049bf0142550ace"
format_stamp: "Formatted at 2026-08-08 10:35:04 on dist-test-slave-9gcw"
server_key: "a92e153fc9ce007fcba9ffab477ced59"
server_key_iv: "35fab7e5652d583eee540690b521c8d9"
server_key_version: "encryptionkey@0"
I20260808 10:35:04.306463 8219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:04.364665 8219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:04.368309 8219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:04.368537 8219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:04.369232 8219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:04.370097 8219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:04.370144 8219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:04.370208 8219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:04.370251 8219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:04.379850 8219 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:43693
I20260808 10:35:04.379896 8363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:43693 every 8 connection(s)
I20260808 10:35:04.380805 8219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:35:04.387323 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 8219
I20260808 10:35:04.387487 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:35:04.387841 31306 external_mini_cluster.cc:1428] Setting key 83043f15e3e42a55e183d5816d56c773
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:35:04.395545 7894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:04.382851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:46023 (local address 127.30.146.190:34763)
0808 10:35:04.383039 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:04.383044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:04.383988 (+ 944us) server_negotiation.cc:407] Connection header received
0808 10:35:04.385335 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:04.385339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:04.385389 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:04.385471 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:04.386619 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.387318 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.388906 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.389274 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.389403 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:04.392172 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:04.392175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:04.392177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:04.392202 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:04.393590 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:04.394117 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:04.394119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:04.394120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:04.394194 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:04.394534 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:04.394536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:04.394537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:04.394710 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:04.394786 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:04.395297 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 10:35:04.395425 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 10:35:04.396498 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:04.383337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:46023)
0808 10:35:04.383826 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:04.383869 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:04.385067 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:04.385623 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:04.385629 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:04.386009 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:04.386476 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:04.386489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.387505 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.387509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:04.388755 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:04.388767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.389442 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.389445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:04.389664 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:04.390468 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:04.390486 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:04.392010 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:04.393713 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:04.393718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:04.393730 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:04.394018 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:04.394290 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:04.394293 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:04.394295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:04.394445 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:04.394815 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:04.394821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:04.395064 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 10:35:04.395244 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 10:35:04.395534 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":116,"threads_started":1}
I20260808 10:35:04.397804 8364 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34763
I20260808 10:35:04.398061 8364 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:04.398626 8364 heartbeater.cc:511] Master 127.30.146.190:34763 requested a full tablet report, sending...
I20260808 10:35:04.399758 7850 ts_manager.cc:194] Registered new tserver with Master: d232722170904b12a049bf0142550ace (127.30.146.131:43693)
I20260808 10:35:04.400437 7850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:46023
I20260808 10:35:04.402159 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab.
Entry for principal oryx/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab.
WARNING: no policy specified for oryx/127.30.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab.
Entry for principal oryx/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab.
WARNING: no policy specified for oryx/127.30.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab.
Entry for principal oryx/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab.
I20260808 10:35:04.490270 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 7922
I20260808 10:35:04.502080 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 8065
I20260808 10:35:04.510222 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 8219
I20260808 10:35:04.516396 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 7812
I20260808 10:35:04.525738 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:34763
--webserver_interface=127.30.146.190
--webserver_port=42915
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:34763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:35:04.643431 8375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:04.643730 8375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:04.643771 8375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:04.647957 8375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:35:04.648037 8375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:04.648057 8375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:04.648074 8375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:35:04.648090 8375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:35:04.653653 8375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:34763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:34763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=42915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:04.654750 8375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:04.655771 8375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:04.664551 8382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:04.664692 8375 server_base.cc:1034] running on GCE node
W20260808 10:35:04.664675 8381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:04.664600 8384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:04.665547 8375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:04.666725 8375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:04.667910 8375 hybrid_clock.cc:648] HybridClock initialized: now 1786185304667896 us; error 28 us; skew 500 ppm
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:04.671275 8375 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:35:04.672281 8375 webserver.cc:460] Webserver started at http://127.30.146.190:42915/ using document root <none> and password file <none>
I20260808 10:35:04.672828 8375 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:04.672904 8375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:04.676961 8375 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260808 10:35:04.679380 8391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:04.680784 8375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:35:04.680932 8375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "5b71f6a8dedc4ecb8672f95101055650"
format_stamp: "Formatted at 2026-08-08 10:35:03 on dist-test-slave-9gcw"
server_key: "f750a231ca924a9ca0d50a2f24f4b51b"
server_key_iv: "721b218f695f06adb033ee573b4339ff"
server_key_version: "encryptionkey@0"
I20260808 10:35:04.681574 8375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:04.720803 8375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:04.724817 8375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:04.725063 8375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:04.732904 8375 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:34763
I20260808 10:35:04.732911 8448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:34763 every 8 connection(s)
I20260808 10:35:04.734143 8375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:35:04.738749 8449 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:35:04.740507 8449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: Bootstrap starting.
I20260808 10:35:04.743994 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 8375
I20260808 10:35:04.744488 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:44991
--local_ip_for_outbound_sockets=127.30.146.129
--tserver_master_addrs=127.30.146.190:34763
--webserver_port=40515
--webserver_interface=127.30.146.129
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.146.129
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:35:04.759385 8449 log.cc:824] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: Log is configured to *not* fsync() on all Append() calls
I20260808 10:35:04.766666 8449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:35:04.767113 8449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: Bootstrap complete.
I20260808 10:35:04.771626 8449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } }
I20260808 10:35:04.772058 8449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b71f6a8dedc4ecb8672f95101055650, State: Initialized, Role: FOLLOWER
I20260808 10:35:04.772612 8449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } }
I20260808 10:35:04.772776 8449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:35:04.772872 8449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:35:04.772998 8449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:35:04.775166 8449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } }
I20260808 10:35:04.775512 8449 leader_election.cc:302] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b71f6a8dedc4ecb8672f95101055650; no voters:
I20260808 10:35:04.775883 8449 leader_election.cc:288] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:35:04.776044 8453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:35:04.776291 8453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [term 2 LEADER]: Becoming Leader. State: Replica: 5b71f6a8dedc4ecb8672f95101055650, State: Running, Role: LEADER
I20260808 10:35:04.776587 8453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } }
I20260808 10:35:04.777179 8449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:35:04.777995 8454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5b71f6a8dedc4ecb8672f95101055650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } } }
I20260808 10:35:04.778143 8454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [sys.catalog]: This master's current role is: LEADER
I20260808 10:35:04.778470 8455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b71f6a8dedc4ecb8672f95101055650. Latest consensus state: current_term: 2 leader_uuid: "5b71f6a8dedc4ecb8672f95101055650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b71f6a8dedc4ecb8672f95101055650" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34763 } } }
I20260808 10:35:04.778563 8455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650 [sys.catalog]: This master's current role is: LEADER
I20260808 10:35:04.779269 8462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:35:04.782740 8462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:35:04.783414 8462 catalog_manager.cc:1259] Loaded cluster ID: 5ed84649485948298d320779457f40e2
I20260808 10:35:04.783478 8462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:35:04.785920 8462 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:35:04.786618 8462 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5b71f6a8dedc4ecb8672f95101055650: Loaded TSK: 0
I20260808 10:35:04.787827 8462 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:35:04.883809 8451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:04.884090 8451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:04.884135 8451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:04.888608 8451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:04.888690 8451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:04.888751 8451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:35:04.894455 8451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.129.keytab
--principal=oryx/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:44991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=40515
--tserver_master_addrs=127.30.146.190:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:04.895608 8451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:04.896763 8451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:04.908129 8480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:04.908175 8477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:04.908245 8478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:04.908710 8451 server_base.cc:1034] running on GCE node
I20260808 10:35:04.909228 8451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:04.909929 8451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:04.911130 8451 hybrid_clock.cc:648] HybridClock initialized: now 1786185304911114 us; error 37 us; skew 500 ppm
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185304, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:04.914417 8451 init.cc:377] Logged in from keytab as oryx/127.30.146.129@KRBTEST.COM (short username oryx)
I20260808 10:35:04.915452 8451 webserver.cc:460] Webserver started at http://127.30.146.129:40515/ using document root <none> and password file <none>
I20260808 10:35:04.916043 8451 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:04.916121 8451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:04.920497 8451 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:35:04.922966 8488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:04.924690 8451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:35:04.924849 8451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "9ae6458bbdce4d4097ff597dac075295"
format_stamp: "Formatted at 2026-08-08 10:35:03 on dist-test-slave-9gcw"
server_key: "a646e6e7d1b070aae97f61edc03f9e7f"
server_key_iv: "e94e95699bb5c886ed8a39cc43a6583f"
server_key_version: "encryptionkey@0"
I20260808 10:35:04.925657 8451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:04.955838 8451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:04.960218 8451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:04.960424 8451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:04.961364 8451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:04.962364 8451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:04.962431 8451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:04.962486 8451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:04.962512 8451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:04.972667 8451 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:44991
I20260808 10:35:04.972711 8617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:44991 every 8 connection(s)
I20260808 10:35:04.973752 8451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:35:04.974493 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 8451
I20260808 10:35:04.975023 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:33557
--local_ip_for_outbound_sockets=127.30.146.130
--tserver_master_addrs=127.30.146.190:34763
--webserver_port=43417
--webserver_interface=127.30.146.130
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.146.130
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:35:04.991006 8620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:04.976342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:55635 (local address 127.30.146.190:34763)
0808 10:35:04.977018 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:04.977041 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:04.977605 (+ 564us) server_negotiation.cc:407] Connection header received
0808 10:35:04.979848 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:04.979859 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:04.980180 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:04.980502 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:04.982249 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.983242 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.985248 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.985754 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.985883 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:04.990238 (+ 4355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:55635: BlockingRecv error: recv got EOF from 127.30.146.129:55635 (error 108)
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":145,"threads_started":1}
I20260808 10:35:04.991042 8621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:04.976468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:55635)
0808 10:35:04.977406 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:04.977468 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:04.979410 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:04.980632 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:04.980644 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:04.981259 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:04.982036 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:04.982061 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.983409 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.983416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:04.985063 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:04.985079 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.985945 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.985950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:04.986205 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:04.986919 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:04.986946 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:04.990014 (+ 3068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":140,"threads_started":1}
W20260808 10:35:04.991742 8618 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:34763 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.146.190:34763: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:04.998034 8621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:04.992239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:44373)
0808 10:35:04.992364 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:04.992379 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:04.992452 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:04.992727 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:04.992729 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:04.992876 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:04.992980 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:04.992985 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.994060 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.994062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:04.994672 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:04.994682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.995335 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.995338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:04.995414 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:04.995962 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:04.995974 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:04.997924 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:35:04.998451 8620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:04.992337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:44373 (local address 127.30.146.190:34763)
0808 10:35:04.992453 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:04.992457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:04.992474 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:35:04.992521 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:04.992523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:04.992569 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:04.992665 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:04.993090 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.993941 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.994789 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:04.995161 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:04.995295 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:04.998100 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:44373: BlockingRecv error: recv got EOF from 127.30.146.129:44373 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:04 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:05.004142 8621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:04.998593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:45945)
0808 10:35:04.998838 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:04.998850 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:04.998944 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:04.999178 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:04.999181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:04.999356 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:04.999429 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:04.999434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.000160 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.000164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.000686 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.000694 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.001172 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.001176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.001270 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.001782 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:05.001794 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:05.003972 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184}
W20260808 10:35:05.004251 8620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:04.998838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:45945 (local address 127.30.146.190:34763)
0808 10:35:04.998968 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:04.998975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:04.998991 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:35:04.999031 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:04.999033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:04.999080 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:04.999163 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:04.999543 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.000048 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.000824 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.001037 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.001142 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.004132 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:45945: BlockingRecv error: recv got EOF from 127.30.146.129:45945 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 10:35:05.004522 8618 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:35:05.110110 8622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:05.110460 8622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:05.110541 8622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:05.114836 8622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:05.114938 8622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:05.115041 8622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:35:05.121479 8622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.130.keytab
--principal=oryx/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:33557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=43417
--tserver_master_addrs=127.30.146.190:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:05.122871 8622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:05.124081 8622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:05.134905 8632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:05.135003 8628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:05.134917 8629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:05.135196 8622 server_base.cc:1034] running on GCE node
I20260808 10:35:05.135946 8622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:05.136796 8622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:05.138005 8622 hybrid_clock.cc:648] HybridClock initialized: now 1786185305137954 us; error 64 us; skew 500 ppm
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185305, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:05.141417 8622 init.cc:377] Logged in from keytab as oryx/127.30.146.130@KRBTEST.COM (short username oryx)
I20260808 10:35:05.142372 8622 webserver.cc:460] Webserver started at http://127.30.146.130:43417/ using document root <none> and password file <none>
I20260808 10:35:05.142954 8622 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:05.143065 8622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:05.147199 8622 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:35:05.149403 8642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:05.150691 8622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:35:05.150853 8622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "386d19185ccd4204b394130bb4e28a62"
format_stamp: "Formatted at 2026-08-08 10:35:04 on dist-test-slave-9gcw"
server_key: "cfa609fafeca45665ab2185408b71f67"
server_key_iv: "dee32b1f0d86ac29e6304b7ee60a625b"
server_key_version: "encryptionkey@0"
I20260808 10:35:05.151391 8622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:05.165421 8622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:05.169023 8622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:05.169296 8622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:05.169968 8622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:05.170905 8622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:05.170981 8622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:05.171059 8622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:05.171103 8622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:05.181588 8622 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:33557
I20260808 10:35:05.181610 8772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:33557 every 8 connection(s)
I20260808 10:35:05.182761 8622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:35:05.184391 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 8622
I20260808 10:35:05.185125 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:43693
--local_ip_for_outbound_sockets=127.30.146.131
--tserver_master_addrs=127.30.146.190:34763
--webserver_port=46329
--webserver_interface=127.30.146.131
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.146.131
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:35:05.197325 8620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:05.185413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:42029 (local address 127.30.146.190:34763)
0808 10:35:05.185605 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:05.185610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:05.186199 (+ 589us) server_negotiation.cc:407] Connection header received
0808 10:35:05.188066 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:05.188070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:05.188136 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:05.188226 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:05.189970 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.190543 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.192639 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.192872 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.192958 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.197190 (+ 4232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:42029: BlockingRecv error: recv got EOF from 127.30.146.130:42029 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:35:05.198007 8775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:05.185073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:42029)
0808 10:35:05.185999 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:05.186065 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:05.187836 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:05.188402 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:05.188409 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:05.188976 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:05.189776 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.189801 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.190706 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.190712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.192497 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.192514 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.193048 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.193056 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.193395 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.194094 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:05.194133 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:05.197022 (+ 2889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":147,"threads_started":1}
W20260808 10:35:05.198554 8773 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:34763 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.146.190:34763: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:05.204357 8775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:05.199059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:58723)
0808 10:35:05.199163 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:05.199178 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:05.199274 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:05.199506 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:05.199509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:05.199678 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:05.199772 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.199778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.200497 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.200500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.201054 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.201063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.201558 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.201561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.201657 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.202164 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:05.202177 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:05.204220 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:35:05.204507 8620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:05.199159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:58723 (local address 127.30.146.190:34763)
0808 10:35:05.199293 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:05.199298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:05.199315 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:35:05.199360 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:05.199363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:05.199409 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:05.199490 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:05.199885 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.200387 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.201207 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.201435 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.201555 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.204361 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:58723: BlockingRecv error: recv got EOF from 127.30.146.130:58723 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:05.210074 8775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:05.205133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:49205)
0808 10:35:05.205289 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:05.205304 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:05.205386 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:05.205629 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:05.205632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:05.205801 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:05.205877 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.205882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.206599 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.206604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.207119 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.207127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.207620 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.207624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.207705 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.208190 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:05.208202 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:05.209931 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 10:35:05.210150 8620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:05.205224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:49205 (local address 127.30.146.190:34763)
0808 10:35:05.205342 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:05.205346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:05.205366 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:35:05.205485 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:05.205488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:05.205534 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:05.205710 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:05.205983 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.206489 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.207242 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.207509 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.207626 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.210038 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:49205: BlockingRecv error: recv got EOF from 127.30.146.130:49205 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:35:05.210572 8773 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:35:05.308781 8776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:05.309093 8776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:05.309237 8776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:05.313443 8776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:05.313525 8776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:05.313612 8776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:35:05.319068 8776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/krb5kdc/oryx_127.30.146.131.keytab
--principal=oryx/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:43693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=46329
--tserver_master_addrs=127.30.146.190:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:05.320236 8776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:05.321394 8776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:05.332039 8782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:05.332279 8783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:05.332039 8785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:05.332556 8776 server_base.cc:1034] running on GCE node
I20260808 10:35:05.333045 8776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:05.333745 8776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:05.334944 8776 hybrid_clock.cc:648] HybridClock initialized: now 1786185305334898 us; error 71 us; skew 500 ppm
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185305, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:05.338124 8776 init.cc:377] Logged in from keytab as oryx/127.30.146.131@KRBTEST.COM (short username oryx)
I20260808 10:35:05.339131 8776 webserver.cc:460] Webserver started at http://127.30.146.131:46329/ using document root <none> and password file <none>
I20260808 10:35:05.339742 8776 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:05.339859 8776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:05.343964 8776 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:35:05.346338 8793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:05.347750 8776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:35:05.347889 8776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "d232722170904b12a049bf0142550ace"
format_stamp: "Formatted at 2026-08-08 10:35:04 on dist-test-slave-9gcw"
server_key: "a92e153fc9ce007fcba9ffab477ced59"
server_key_iv: "35fab7e5652d583eee540690b521c8d9"
server_key_version: "encryptionkey@0"
I20260808 10:35:05.348573 8776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:05.364245 8776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:05.367990 8776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:05.368238 8776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:05.368903 8776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:05.369833 8776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:05.369904 8776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:05.369982 8776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:05.370023 8776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:05.379904 8776 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:43693
I20260808 10:35:05.379921 8921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:43693 every 8 connection(s)
I20260808 10:35:05.381014 8776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:35:05.383997 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 8776
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:35:05.394598 8620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:05.382972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:37217 (local address 127.30.146.190:34763)
0808 10:35:05.383134 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:05.383138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:05.384141 (+ 1003us) server_negotiation.cc:407] Connection header received
0808 10:35:05.386184 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:05.386190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:05.386314 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:05.386416 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:05.387987 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.388678 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.390023 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.390409 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.390536 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.394465 (+ 3929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:37217: BlockingRecv error: recv got EOF from 127.30.146.131:37217 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:35:05.395308 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:05.385222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57412 (local address 127.30.146.190:34763)
0808 10:35:05.385541 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:05.385545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:05.385560 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:35:05.385605 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:05.385608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:05.385661 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:05.385764 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:05.386653 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.387475 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.388562 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.388856 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.388963 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.390636 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:05.390642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:05.390661 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:05.390697 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:05.392722 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:05.393312 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:05.393318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:05.393324 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:05.393492 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:05.393843 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:05.393846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:05.393848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:05.394213 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:05.394492 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:05.394989 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 10:35:05.395183 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":97,"threads_started":1}
I20260808 10:35:05.395378 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:05.383401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:37217)
0808 10:35:05.383947 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:05.384013 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:05.385975 (+ 1962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:05.386643 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:05.386653 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:05.387223 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:05.387842 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.387858 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.388833 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.388836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.389888 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.389898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.390585 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.390590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.390900 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.391487 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:05.391510 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:05.394361 (+ 2851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":113,"threads_started":1}
W20260808 10:35:05.396226 8922 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:34763 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.146.190:34763: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:05.401881 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:05.396691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:33753)
0808 10:35:05.396819 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:05.396835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:05.396927 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:05.397224 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:05.397227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:05.397391 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:05.397510 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.397517 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.398393 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.398395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.398959 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.398967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.399388 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.399390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.399456 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.399967 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:05.399978 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:05.401781 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 10:35:05.402014 8926 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:05.396863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:33753 (local address 127.30.146.190:34763)
0808 10:35:05.397005 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:05.397009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:05.397021 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:35:05.397058 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:05.397060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:05.397092 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:05.397208 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:05.397644 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.398296 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.399056 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.399292 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.399369 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.401905 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:33753: BlockingRecv error: recv got EOF from 127.30.146.131:33753 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:05 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:05.407158 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:05.402477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:44575)
0808 10:35:05.402583 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:05.402596 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:05.402668 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:05.403009 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:05.403012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:05.403197 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:05.403267 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.403272 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.403975 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.403979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.404486 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:05.404494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.404942 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.404945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:05.405022 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.405473 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:05.405483 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:05.407009 (+ 1526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 10:35:05.407226 8926 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:05.402574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:44575 (local address 127.30.146.190:34763)
0808 10:35:05.402835 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:05.402838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:05.402848 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:35:05.402877 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:05.402879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:05.402910 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:05.402993 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:05.403381 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.403868 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.404602 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:05.404836 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:05.404942 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:05.407115 (+ 2173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:44575: BlockingRecv error: recv got EOF from 127.30.146.131:44575 (error 108)
Metrics: {"server-negotiator.queue_time_us":184}
W20260808 10:35:05.407428 8922 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 10:35:06 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:06 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:06.017302 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:06.005330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:46637)
0808 10:35:06.005612 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:06.005625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:06.005719 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:06.005978 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:06.005980 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:06.006169 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:06.006289 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:06.006294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.007135 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.007138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:06.007813 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:06.007824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.008343 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.008346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:06.008417 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:06.014937 (+ 6520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:06.014952 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:06.017106 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":91,"threads_started":1}
W20260808 10:35:06.017712 8928 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:06.005380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:46637 (local address 127.30.146.190:34763)
0808 10:35:06.005741 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:06.005745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:06.005784 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:35:06.005829 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:06.005831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:06.005874 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:06.005942 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:06.006402 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.007022 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.007954 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.008214 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.008312 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:06.017465 (+ 9153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:46637: BlockingRecv error: recv got EOF from 127.30.146.129:46637 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":97,"threads_started":1}
Aug 08 10:35:06 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:06 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:06.217137 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:06.211329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:39501)
0808 10:35:06.211623 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:06.211637 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:06.211739 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:06.212007 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:06.212010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:06.212196 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:06.212320 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:06.212325 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.213171 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.213173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:06.213822 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:06.213833 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.214439 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.214442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:06.214529 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:06.215126 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:06.215139 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:06.217011 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
W20260808 10:35:06.217527 8928 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:06.211425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:39501 (local address 127.30.146.190:34763)
0808 10:35:06.211596 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:06.211600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:06.211698 (+ 98us) server_negotiation.cc:407] Connection header received
0808 10:35:06.211813 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:06.211815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:06.211871 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:06.211960 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:06.212452 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.213041 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.213953 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.214310 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.214422 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:06.217238 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:39501: BlockingRecv error: recv got EOF from 127.30.146.130:39501 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 10:35:06 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:06 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:06.413571 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:06.408049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:43551)
0808 10:35:06.408322 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:06.408336 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:06.408434 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:06.408671 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:06.408673 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:06.408887 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:06.408988 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:06.408993 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.409796 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.409798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:06.410363 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:06.410371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.410825 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.410827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:06.410887 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:06.411424 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:06.411435 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:06.413442 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":71,"threads_started":1}
W20260808 10:35:06.413816 8928 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:06.408135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:43551 (local address 127.30.146.190:34763)
0808 10:35:06.408336 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:06.408340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:06.408410 (+ 70us) server_negotiation.cc:407] Connection header received
0808 10:35:06.408514 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:06.408516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:06.408564 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:06.408634 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:06.409090 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.409633 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.410464 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:06.410727 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:06.410810 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:06.413665 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:43551: BlockingRecv error: recv got EOF from 127.30.146.131:43551 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 08 10:35:07 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:07 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:07.024567 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:07.018386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:46353)
0808 10:35:07.018769 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:07.018782 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:07.018914 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:07.019255 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:07.019258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:07.019495 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:07.019624 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:07.019630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.020540 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.020543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:07.021112 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:07.021120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.021632 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.021635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:07.021696 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:07.022320 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:07.022332 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:07.024436 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":91,"threads_started":1}
W20260808 10:35:07.024798 8932 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:07.018488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:46353 (local address 127.30.146.190:34763)
0808 10:35:07.018848 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:07.018853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:07.018870 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:35:07.019034 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:07.019037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:07.019099 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:07.019195 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:07.019753 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.020429 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.021282 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.021527 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.021622 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:07.024622 (+ 3000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:46353: BlockingRecv error: recv got EOF from 127.30.146.129:46353 (error 108)
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":138,"threads_started":1}
Aug 08 10:35:07 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:07 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:07.224025 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:07.218043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:54755)
0808 10:35:07.218348 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:07.218362 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:07.218479 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:07.218810 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:07.218813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:07.219000 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:07.219096 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:07.219101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.219925 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.219929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:07.220582 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:07.220593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.221083 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.221086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:07.221225 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:07.221788 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:07.221800 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:07.223881 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":101,"threads_started":1}
W20260808 10:35:07.224241 8932 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:07.218186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:54755 (local address 127.30.146.190:34763)
0808 10:35:07.218333 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:07.218337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:07.218475 (+ 138us) server_negotiation.cc:407] Connection header received
0808 10:35:07.218553 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:07.218555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:07.218608 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:07.218778 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:07.219220 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.219786 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.220713 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.220950 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.221074 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:07.224070 (+ 2996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:54755: BlockingRecv error: recv got EOF from 127.30.146.130:54755 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 10:35:07 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:07 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:07.420867 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:07.414410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:42387)
0808 10:35:07.414726 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:07.414741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:07.414842 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:07.415143 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:07.415147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:07.415375 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:07.415495 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:07.415502 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.416384 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.416387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:07.417112 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:07.417123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.417674 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.417677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:07.417751 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:07.418274 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:07.418287 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:07.420704 (+ 2417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":118,"threads_started":1}
W20260808 10:35:07.421108 8932 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:07.414535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:42387 (local address 127.30.146.190:34763)
0808 10:35:07.414676 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:07.414680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:07.414809 (+ 129us) server_negotiation.cc:407] Connection header received
0808 10:35:07.414923 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:07.414925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:07.414973 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:07.415039 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:07.415717 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.416267 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.417279 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:07.417493 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:07.417572 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:07.420951 (+ 3379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:42387: BlockingRecv error: recv got EOF from 127.30.146.131:42387 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 10:35:08 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:08 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:08.032042 8936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:08.025752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:42051)
0808 10:35:08.026020 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:08.026037 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:08.026177 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:08.026480 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:08.026483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:08.026784 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:08.026888 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:08.026895 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.027872 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.027879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:08.028502 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:08.028511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.028987 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.028991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:08.029087 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:08.029711 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:08.029726 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:08.031912 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":71,"threads_started":1}
W20260808 10:35:08.032295 8935 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:08.025714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:42051 (local address 127.30.146.190:34763)
0808 10:35:08.026021 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:08.026028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:08.026120 (+ 92us) server_negotiation.cc:407] Connection header received
0808 10:35:08.026267 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:08.026270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:08.026351 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:08.026463 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:08.027071 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.027669 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.028633 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.028872 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.028982 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:08.032125 (+ 3143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:42051: BlockingRecv error: recv got EOF from 127.30.146.129:42051 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":97,"threads_started":1}
Aug 08 10:35:08 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:08 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:08.231420 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:08.225021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:44841)
0808 10:35:08.225366 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:08.225385 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:08.225496 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:08.225846 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:08.225848 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:08.226067 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:08.226199 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:08.226206 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.227392 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.227398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:08.228093 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:08.228105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.228555 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.228558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:08.228626 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:08.229206 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:08.229219 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:08.231220 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":142,"threads_started":1}
W20260808 10:35:08.231521 8935 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:08.225232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:44841 (local address 127.30.146.190:34763)
0808 10:35:08.225399 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:08.225403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:08.225451 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:35:08.225638 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:08.225642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:08.225694 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:08.225780 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:08.226456 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.227257 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.228235 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.228453 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.228536 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:08.231408 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:44841: BlockingRecv error: recv got EOF from 127.30.146.130:44841 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 10:35:08 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:08 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:08.427510 8938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:08.421751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:39555)
0808 10:35:08.422029 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:08.422043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:08.422145 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:08.422413 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:08.422415 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:08.422602 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:08.422701 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:08.422706 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.423496 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.423499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:08.424081 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:08.424092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.424579 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.424581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:08.424650 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:08.425200 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:08.425212 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:08.427387 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":77,"threads_started":1}
W20260808 10:35:08.427661 8935 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:08.421899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:39555 (local address 127.30.146.190:34763)
0808 10:35:08.422066 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:08.422070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:08.422089 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:35:08.422245 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:08.422248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:08.422299 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:08.422376 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:08.422818 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.423369 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.424249 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:08.424474 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:08.424562 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:08.427539 (+ 2977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:39555: BlockingRecv error: recv got EOF from 127.30.146.131:39555 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 08 10:35:09 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:09 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:09.040717 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:09.033113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:53555)
0808 10:35:09.033440 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:09.033455 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:09.033558 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:09.034102 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:09.034106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:09.034332 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:09.034472 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:09.034480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.035554 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.035557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:09.036257 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:09.036268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.036896 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.036899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:09.036992 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:09.037597 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:09.037617 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:09.040567 (+ 2950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":97,"threads_started":1}
W20260808 10:35:09.040905 8940 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:09.033258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:53555 (local address 127.30.146.190:34763)
0808 10:35:09.033519 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:09.033523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:09.033546 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:35:09.033701 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:09.033705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:09.033839 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:09.033980 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:09.034762 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.035405 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.036396 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.036668 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.036818 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:09.040753 (+ 3935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:53555: BlockingRecv error: recv got EOF from 127.30.146.129:53555 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":87,"threads_started":1}
Aug 08 10:35:09 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:09 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:09.238260 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:09.232390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:47689)
0808 10:35:09.232691 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:09.232704 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:09.232835 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:09.233105 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:09.233108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:09.233372 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:09.233470 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:09.233476 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.234297 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.234302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:09.234928 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:09.234937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.235425 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.235428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:09.235516 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:09.236104 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:09.236117 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:09.238073 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":97,"threads_started":1}
W20260808 10:35:09.238309 8940 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:09.232443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:47689 (local address 127.30.146.190:34763)
0808 10:35:09.232603 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:09.232607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:09.232783 (+ 176us) server_negotiation.cc:407] Connection header received
0808 10:35:09.232940 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:09.232943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:09.232999 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:09.233083 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:09.233598 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.234162 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.235049 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.235293 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.235419 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:09.238204 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:47689: BlockingRecv error: recv got EOF from 127.30.146.130:47689 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 10:35:09 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:09 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:09.434927 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:09.428480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:49359)
0808 10:35:09.428771 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:09.428789 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:09.428909 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:09.429236 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:09.429239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:09.429441 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:09.429572 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:09.429579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.430352 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.430355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:09.431071 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:09.431082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.431525 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.431527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:09.431591 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:09.432204 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:09.432221 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:09.434746 (+ 2525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":96,"threads_started":1}
W20260808 10:35:09.435117 8940 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:09.428637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:49359 (local address 127.30.146.190:34763)
0808 10:35:09.428802 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:09.428805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:09.428877 (+ 72us) server_negotiation.cc:407] Connection header received
0808 10:35:09.428998 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:09.429000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:09.429048 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:09.429128 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:09.429703 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.430244 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.431202 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:09.431428 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:09.431504 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:09.434964 (+ 3460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:49359: BlockingRecv error: recv got EOF from 127.30.146.131:49359 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 10:35:10 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:10 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:10.047974 8943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:10.041843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:50007)
0808 10:35:10.042367 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:10.042380 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:10.042472 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:10.042886 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:10.042889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:10.043169 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:10.043290 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:10.043297 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.044082 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.044087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:10.044699 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:10.044708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.045228 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.045232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:10.045350 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:10.045870 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:10.045882 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:10.047766 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":129,"threads_started":1}
W20260808 10:35:10.048056 8944 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:10.041972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:50007 (local address 127.30.146.190:34763)
0808 10:35:10.042255 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:10.042259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:10.042463 (+ 204us) server_negotiation.cc:407] Connection header received
0808 10:35:10.042553 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:10.042556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:10.042669 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:10.042870 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:10.043409 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.043975 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.044828 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.045060 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.045194 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:10.047922 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:50007: BlockingRecv error: recv got EOF from 127.30.146.129:50007 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":69,"threads_started":1}
Aug 08 10:35:10 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:10 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:10.245280 8945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:10.239255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:34657)
0808 10:35:10.239566 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:10.239580 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:10.239732 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:10.240007 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:10.240009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:10.240212 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:10.240311 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:10.240316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.241199 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.241202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:10.241833 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:10.241844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.242338 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.242341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:10.242406 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:10.242944 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:10.242958 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:10.245071 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":110,"threads_started":1}
W20260808 10:35:10.245399 8944 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:10.239364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:34657 (local address 127.30.146.190:34763)
0808 10:35:10.239549 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:10.239553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:10.239638 (+ 85us) server_negotiation.cc:407] Connection header received
0808 10:35:10.239829 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:10.239831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:10.239886 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:10.239973 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:10.240428 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.241041 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.241957 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.242227 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.242317 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:10.245266 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:34657: BlockingRecv error: recv got EOF from 127.30.146.130:34657 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 10:35:10 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:10 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:10.442212 8946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:10.435898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:41895)
0808 10:35:10.436406 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:10.436422 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:10.436541 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:10.436842 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:10.436845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:10.437067 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:10.437233 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:10.437281 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.438305 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.438308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:10.438914 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:10.438923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.439389 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.439392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:10.439472 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:10.440090 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:10.440110 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:10.442085 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":93,"threads_started":1}
W20260808 10:35:10.442472 8944 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:10.436156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:41895 (local address 127.30.146.190:34763)
0808 10:35:10.436354 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:10.436358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:10.436498 (+ 140us) server_negotiation.cc:407] Connection header received
0808 10:35:10.436657 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:10.436660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:10.436716 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:10.436815 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:10.437411 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.438183 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.439027 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:10.439278 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:10.439368 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:10.442310 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:41895: BlockingRecv error: recv got EOF from 127.30.146.131:41895 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 10:35:11 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:11 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:11.058495 8947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:11.049173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:39345)
0808 10:35:11.049454 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:11.049475 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:11.049572 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:11.049912 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:11.049914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:11.050102 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:11.050233 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:11.050239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.051037 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.051040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:11.051662 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:11.051674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.052232 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.052235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:11.052300 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:11.056251 (+ 3951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:11.056264 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:11.058297 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":95,"threads_started":1}
W20260808 10:35:11.058581 8948 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:11.049382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:39345 (local address 127.30.146.190:34763)
0808 10:35:11.049657 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:11.049663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:11.049690 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:35:11.049733 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:11.049735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:11.049778 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:11.049874 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:11.050357 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.050929 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.051790 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.052111 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.052236 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:11.058450 (+ 6214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:39345: BlockingRecv error: recv got EOF from 127.30.146.129:39345 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
Aug 08 10:35:11 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:11 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:11.251981 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:11.246092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:54255)
0808 10:35:11.246344 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:11.246357 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:11.246492 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:11.246764 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:11.246767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:11.246973 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:11.247066 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:11.247071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.247851 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.247856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:11.248413 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:11.248421 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.248876 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.248880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:11.248968 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:11.249632 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:11.249645 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:11.251805 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
W20260808 10:35:11.252050 8948 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:11.246129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:54255 (local address 127.30.146.190:34763)
0808 10:35:11.246273 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:11.246277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:11.246432 (+ 155us) server_negotiation.cc:407] Connection header received
0808 10:35:11.246602 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:11.246604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:11.246651 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:11.246730 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:11.247180 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.247744 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.248535 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.248763 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.248862 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:11.251940 (+ 3078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:54255: BlockingRecv error: recv got EOF from 127.30.146.130:54255 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 10:35:11 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:11 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:11.449041 8950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:11.443060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:51329)
0808 10:35:11.443372 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:11.443400 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:11.443531 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:11.443822 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:11.443825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:11.444048 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:11.444142 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:11.444148 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.444969 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.444973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:11.445596 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:11.445605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.446060 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.446064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:11.446140 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:11.446802 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:11.446815 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:11.448869 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":117,"threads_started":1}
W20260808 10:35:11.449136 8948 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:11.443162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:51329 (local address 127.30.146.190:34763)
0808 10:35:11.443350 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:11.443355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:11.443475 (+ 120us) server_negotiation.cc:407] Connection header received
0808 10:35:11.443635 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:11.443637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:11.443692 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:11.443800 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:11.444273 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.444839 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.445718 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:11.445947 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:11.446056 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:11.449009 (+ 2953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:51329: BlockingRecv error: recv got EOF from 127.30.146.131:51329 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 08 10:35:12 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:12 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:12.065474 8951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:12.059440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:50319)
0808 10:35:12.059802 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:12.059818 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:12.059921 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:12.060189 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:12.060192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:12.060412 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:12.060520 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:12.060527 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.061403 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.061406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:12.062044 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:12.062054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.062564 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.062567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:12.062641 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:12.063211 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:12.063225 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:12.065342 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":78,"threads_started":1}
W20260808 10:35:12.065790 8953 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:12.059563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:50319 (local address 127.30.146.190:34763)
0808 10:35:12.059888 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:12.059892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:12.059907 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:35:12.059997 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:12.059999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:12.060040 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:12.060115 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:12.060644 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.061275 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.062183 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.062440 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.062539 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:12.065527 (+ 2988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:50319: BlockingRecv error: recv got EOF from 127.30.146.129:50319 (error 108)
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
Aug 08 10:35:12 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:12 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:12.259397 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:12.253006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:48369)
0808 10:35:12.253436 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:12.253451 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:12.253564 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:12.253828 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:12.253831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:12.254092 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:12.254203 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:12.254210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.255034 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.255038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:12.255651 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:12.255660 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.256115 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.256118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:12.256204 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:12.256776 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:12.256795 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:12.259268 (+ 2473us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":161,"threads_started":1}
W20260808 10:35:12.259559 8953 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:12.253091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:48369 (local address 127.30.146.190:34763)
0808 10:35:12.253334 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:12.253339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:12.253551 (+ 212us) server_negotiation.cc:407] Connection header received
0808 10:35:12.253643 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:12.253645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:12.253693 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:12.253769 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:12.254316 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.254925 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.255774 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.255995 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.256098 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:12.259404 (+ 3306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:48369: BlockingRecv error: recv got EOF from 127.30.146.130:48369 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
Aug 08 10:35:12 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:12 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:12.455776 8955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:12.450016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:32781)
0808 10:35:12.450347 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:12.450360 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:12.450460 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:12.450697 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:12.450699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:12.450901 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:12.451005 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:12.451010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.451818 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.451821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:12.452419 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:12.452428 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.452924 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.452927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:12.453030 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:12.453652 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:12.453664 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:12.455657 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":94,"threads_started":1}
W20260808 10:35:12.456084 8953 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:12.450049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:32781 (local address 127.30.146.190:34763)
0808 10:35:12.450239 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:12.450245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:12.450430 (+ 185us) server_negotiation.cc:407] Connection header received
0808 10:35:12.450540 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:12.450542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:12.450591 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:12.450666 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:12.451106 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.451671 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.452527 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:12.452780 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:12.452910 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:12.455794 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:32781: BlockingRecv error: recv got EOF from 127.30.146.131:32781 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 10:35:13 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:13 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:13.072234 8956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:13.066373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:37245)
0808 10:35:13.066711 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:13.066731 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:13.066846 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:13.067104 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:13.067106 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:13.067303 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:13.067406 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:13.067411 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.068219 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.068222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:13.068766 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:13.068773 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.069307 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.069310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:13.069374 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:13.069984 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:13.069996 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:13.072072 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":97,"threads_started":1}
W20260808 10:35:13.072386 8957 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:13.066554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:37245 (local address 127.30.146.190:34763)
0808 10:35:13.066828 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:13.066832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:13.066848 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:35:13.066926 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:13.066928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:13.066972 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:13.067091 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:13.067517 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.068110 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.068885 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.069166 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.069295 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:13.072241 (+ 2946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:37245: BlockingRecv error: recv got EOF from 127.30.146.129:37245 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
Aug 08 10:35:13 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:13 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:13.266320 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:13.260220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:39139)
0808 10:35:13.260506 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:13.260520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:13.260753 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:13.261074 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:13.261076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:13.261386 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:13.261513 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:13.261521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.262634 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.262638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:13.263230 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:13.263239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.263635 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.263638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:13.263699 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:13.264200 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:13.264213 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:13.266204 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":82,"threads_started":1}
W20260808 10:35:13.266501 8957 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:13.260384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:39139 (local address 127.30.146.190:34763)
0808 10:35:13.260555 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:13.260559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:13.260574 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:35:13.260887 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:13.260890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:13.260946 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:13.261040 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:13.261654 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.262531 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.263334 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.263543 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.263616 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:13.266365 (+ 2749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:39139: BlockingRecv error: recv got EOF from 127.30.146.130:39139 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 10:35:13 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:13 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:13.463112 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:13.456533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:50253)
0808 10:35:13.456874 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:13.456889 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:13.457031 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:13.457390 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:13.457394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:13.457639 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:13.457761 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:13.457769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.458601 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.458608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:13.459235 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:13.459244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.459760 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.459764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:13.459861 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:13.460401 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:13.460413 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:13.462918 (+ 2505us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":102,"threads_started":1}
W20260808 10:35:13.463215 8957 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:13.456650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:50253 (local address 127.30.146.190:34763)
0808 10:35:13.456820 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:13.456826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:13.456967 (+ 141us) server_negotiation.cc:407] Connection header received
0808 10:35:13.457201 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:13.457205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:13.457274 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:13.457367 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:13.457890 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.458482 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.459372 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:13.459627 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:13.459741 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:13.463052 (+ 3311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:50253: BlockingRecv error: recv got EOF from 127.30.146.131:50253 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 10:35:14 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:14 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:14.079058 8960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:14.073260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:50793)
0808 10:35:14.073562 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:14.073579 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:14.073690 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:14.074003 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:14.074006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:14.074217 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:14.074332 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:14.074337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.075133 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.075138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:14.075720 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:14.075728 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.076186 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.076190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:14.076287 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:14.076829 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:14.076841 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:14.078940 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
W20260808 10:35:14.079212 8961 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:14.073346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:50793 (local address 127.30.146.190:34763)
0808 10:35:14.073638 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:14.073643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:14.073657 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:35:14.073833 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:14.073836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:14.073888 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:14.073978 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:14.074449 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.075025 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.075843 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.076076 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.076181 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:14.079077 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:50793: BlockingRecv error: recv got EOF from 127.30.146.129:50793 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":96,"threads_started":1}
Aug 08 10:35:14 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:14 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:14.273268 8962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:14.267141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:54363)
0808 10:35:14.267494 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:14.267508 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:14.267609 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:14.267860 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:14.267862 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:14.268048 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:14.268146 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:14.268151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.268937 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.268940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:14.269582 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:14.269593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.270178 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.270181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:14.270251 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:14.270798 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:14.270810 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:14.273020 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":174,"threads_started":1}
W20260808 10:35:14.273316 8961 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:14.267219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:54363 (local address 127.30.146.190:34763)
0808 10:35:14.267394 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:14.267398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:14.267580 (+ 182us) server_negotiation.cc:407] Connection header received
0808 10:35:14.267693 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:14.267699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:14.267745 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:14.267822 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:14.268252 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.268804 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.269833 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.270055 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.270134 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:14.273199 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:54363: BlockingRecv error: recv got EOF from 127.30.146.130:54363 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 10:35:14 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:14 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:14.469705 8963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:14.463972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:32923)
0808 10:35:14.464236 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:14.464249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:14.464344 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:14.464705 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:14.464707 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:14.464916 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:14.465020 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:14.465030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.465871 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.465874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:14.466454 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:14.466464 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.466916 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.466919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:14.466985 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:14.467511 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:14.467523 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:14.469567 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":98,"threads_started":1}
W20260808 10:35:14.469871 8961 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:14.464059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:32923 (local address 127.30.146.190:34763)
0808 10:35:14.464194 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:14.464198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:14.464312 (+ 114us) server_negotiation.cc:407] Connection header received
0808 10:35:14.464421 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:14.464423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:14.464472 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:14.464547 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:14.465176 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.465740 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.466565 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:14.466785 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:14.466884 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:14.469719 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:32923: BlockingRecv error: recv got EOF from 127.30.146.131:32923 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 10:35:15 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:15 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:15.085749 8964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:15.079971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:34647)
0808 10:35:15.080254 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:15.080268 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:15.080358 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:15.080592 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:15.080594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:15.080778 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:15.080876 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:15.080881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.081730 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.081733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:15.082324 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:15.082332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.082834 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.082837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:15.082900 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:15.083388 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:15.083400 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:15.085598 (+ 2198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
W20260808 10:35:15.085943 8965 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:15.080019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:34647 (local address 127.30.146.190:34763)
0808 10:35:15.080330 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:15.080334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:15.080351 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:35:15.080428 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:15.080431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:15.080475 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:15.080562 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:15.080990 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.081628 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.082442 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.082727 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.082820 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:15.085780 (+ 2960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:34647: BlockingRecv error: recv got EOF from 127.30.146.129:34647 (error 108)
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":96,"threads_started":1}
Aug 08 10:35:15 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:15 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:15.280521 8966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:15.274181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:42633)
0808 10:35:15.274509 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:15.274525 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:15.274677 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:15.274956 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:15.274959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:15.275215 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:15.275309 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:15.275314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.276180 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.276183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:15.276807 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:15.276816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.277342 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.277346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:15.277456 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:15.278107 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:15.278120 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:15.280372 (+ 2252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":89,"threads_started":1}
W20260808 10:35:15.280622 8965 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:15.274325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:42633 (local address 127.30.146.190:34763)
0808 10:35:15.274513 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:15.274521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:15.274618 (+ 97us) server_negotiation.cc:407] Connection header received
0808 10:35:15.274794 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:15.274797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:15.274850 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:15.274934 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:15.275459 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.276006 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.276925 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.277227 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.277337 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:15.280520 (+ 3183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:42633: BlockingRecv error: recv got EOF from 127.30.146.130:42633 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 10:35:15 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:15 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:15.477262 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:15.470590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:54985)
0808 10:35:15.470987 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:15.471008 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:15.471163 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:15.471475 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:15.471478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:15.471801 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:15.471907 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:15.471914 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.472815 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.472821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:15.473487 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:15.473497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.474045 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.474049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:15.474153 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:15.474745 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:15.474758 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:15.477006 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":121,"threads_started":1}
W20260808 10:35:15.477332 8965 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:15.470808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:54985 (local address 127.30.146.190:34763)
0808 10:35:15.470986 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:15.470993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:15.471107 (+ 114us) server_negotiation.cc:407] Connection header received
0808 10:35:15.471284 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:15.471287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:15.471346 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:15.471444 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:15.472049 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.472683 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.473633 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:15.473900 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:15.474020 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:15.477206 (+ 3186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:54985: BlockingRecv error: recv got EOF from 127.30.146.131:54985 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 08 10:35:16 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:16 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:16.098477 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:16.086901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:58431)
0808 10:35:16.087219 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:16.087232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:16.087365 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:16.087611 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:16.087613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:16.087804 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:16.087904 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:16.087909 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.088823 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.088828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:16.089586 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:16.089599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.090209 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.090212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:16.090304 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:16.096021 (+ 5717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:16.096038 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:16.098295 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":132,"threads_started":1}
W20260808 10:35:16.098677 8969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:16.086922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:58431 (local address 127.30.146.190:34763)
0808 10:35:16.087236 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:16.087240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:16.087361 (+ 121us) server_negotiation.cc:407] Connection header received
0808 10:35:16.087441 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:16.087443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:16.087495 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:16.087582 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:16.088050 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.088661 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.089755 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.090028 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.090200 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:16.098497 (+ 8297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:58431: BlockingRecv error: recv got EOF from 127.30.146.129:58431 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":79,"threads_started":1}
Aug 08 10:35:16 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:16 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:16.288237 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:16.281608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:45105)
0808 10:35:16.282012 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:16.282028 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:16.282186 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:16.282488 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:16.282491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:16.282816 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:16.282926 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:16.282933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.283825 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.283831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:16.284474 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:16.284483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.285053 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.285057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:16.285188 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:16.285924 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:16.285939 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:16.288074 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":141,"threads_started":1}
W20260808 10:35:16.288362 8969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:16.281757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:45105 (local address 127.30.146.190:34763)
0808 10:35:16.281966 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:16.281972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:16.282133 (+ 161us) server_negotiation.cc:407] Connection header received
0808 10:35:16.282304 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:16.282307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:16.282365 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:16.282464 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:16.283067 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.283695 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.284610 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.284922 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.285045 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:16.288220 (+ 3175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:45105: BlockingRecv error: recv got EOF from 127.30.146.130:45105 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 08 10:35:16 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:16 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:16.484731 8972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:16.478417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:56141)
0808 10:35:16.478778 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:16.478795 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:16.478917 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:16.479240 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:16.479243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:16.479478 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:16.479581 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:16.479587 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.480492 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.480494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:16.481054 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:16.481062 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.481628 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.481630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:16.481695 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:16.482310 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:16.482325 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:16.484565 (+ 2240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":101,"threads_started":1}
W20260808 10:35:16.485196 8969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:16.478519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:56141 (local address 127.30.146.190:34763)
0808 10:35:16.478642 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:16.478647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:16.478871 (+ 224us) server_negotiation.cc:407] Connection header received
0808 10:35:16.479029 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:16.479033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:16.479099 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:16.479203 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:16.479717 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.480363 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.481234 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:16.481514 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:16.481606 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:16.485035 (+ 3429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:56141: BlockingRecv error: recv got EOF from 127.30.146.131:56141 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 10:35:17 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:17 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:17.105541 8974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:17.099442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:34267)
0808 10:35:17.099754 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:17.099777 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:17.099887 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:17.100197 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:17.100199 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:17.100390 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:17.100502 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:17.100507 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.101374 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.101378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:17.102256 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:17.102266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.102787 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.102790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:17.102855 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:17.103390 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:17.103402 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:17.105409 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":112,"threads_started":1}
W20260808 10:35:17.105731 8975 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:17.099640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:34267 (local address 127.30.146.190:34763)
0808 10:35:17.099933 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:17.099937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:17.099951 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:35:17.099993 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:17.099995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:17.100038 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:17.100127 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:17.100614 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.101256 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.102417 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.102676 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.102761 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:17.105554 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:34267: BlockingRecv error: recv got EOF from 127.30.146.129:34267 (error 108)
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":140,"threads_started":1}
Aug 08 10:35:17 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:17 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:17.297224 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:17.289416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:46313)
0808 10:35:17.289699 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:17.289721 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:17.289881 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:17.290254 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:17.290257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:17.290646 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:17.290813 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:17.290834 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.292073 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.292077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:17.292712 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:17.292722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.293334 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.293340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:17.293468 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:17.294156 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:17.294175 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:17.296970 (+ 2795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
W20260808 10:35:17.297606 8975 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:17.289475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:46313 (local address 127.30.146.190:34763)
0808 10:35:17.289631 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:17.289636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:17.289832 (+ 196us) server_negotiation.cc:407] Connection header received
0808 10:35:17.290002 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:17.290006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:17.290081 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:17.290198 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:17.290994 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.291915 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.292852 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.293127 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.293292 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:17.297438 (+ 4146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:46313: BlockingRecv error: recv got EOF from 127.30.146.130:46313 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 10:35:17 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:17 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:17.492991 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:17.485790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:58155)
0808 10:35:17.486135 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:17.486149 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:17.486254 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:17.486573 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:17.486575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:17.486832 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:17.486946 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:17.486953 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.488274 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.488277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:17.488948 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:17.488960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.489710 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.489713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:17.489790 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:17.490428 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:17.490442 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:17.492796 (+ 2354us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":107,"threads_started":1}
W20260808 10:35:17.493067 8975 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:17.485771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:58155 (local address 127.30.146.190:34763)
0808 10:35:17.485961 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:17.485967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:17.486215 (+ 248us) server_negotiation.cc:407] Connection header received
0808 10:35:17.486362 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:17.486366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:17.486439 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:17.486542 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:17.487101 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.488122 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.489091 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:17.489564 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:17.489694 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:17.492939 (+ 3245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:58155: BlockingRecv error: recv got EOF from 127.30.146.131:58155 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 10:35:18 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:18 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:18.112231 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:18.106418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:42607)
0808 10:35:18.106773 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:18.106787 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:18.106895 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:18.107117 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:18.107119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:18.107308 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:18.107414 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:18.107419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.108205 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.108207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:18.108862 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:18.108873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.109330 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.109333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:18.109399 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:18.110055 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:18.110068 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:18.112105 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":94,"threads_started":1}
W20260808 10:35:18.112452 8980 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:18.106493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:42607 (local address 127.30.146.190:34763)
0808 10:35:18.106772 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:18.106776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:18.106845 (+ 69us) server_negotiation.cc:407] Connection header received
0808 10:35:18.106980 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:18.106982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:18.107027 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:18.107103 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:18.107525 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.108078 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.108980 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.109224 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.109302 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:18.112257 (+ 2955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:42607: BlockingRecv error: recv got EOF from 127.30.146.129:42607 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":65,"threads_started":1}
Aug 08 10:35:18 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:18 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:18.304352 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:18.298291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:51813)
0808 10:35:18.298569 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:18.298585 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:18.298682 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:18.298911 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:18.298912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:18.299101 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:18.299213 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:18.299219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.300117 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.300119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:18.300685 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:18.300693 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.301265 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.301271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:18.301371 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:18.302057 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:18.302072 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:18.304203 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
W20260808 10:35:18.304541 8980 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:18.298485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:51813 (local address 127.30.146.190:34763)
0808 10:35:18.298654 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:18.298658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:18.298673 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:35:18.298751 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:18.298753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:18.298798 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:18.298879 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:18.299352 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.300014 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.300796 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.301037 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.301230 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:18.304380 (+ 3150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:51813: BlockingRecv error: recv got EOF from 127.30.146.130:51813 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 10:35:18 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:18 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:18.499643 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:18.493865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:35347)
0808 10:35:18.494166 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:18.494182 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:18.494325 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:18.494610 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:18.494613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:18.494829 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:18.494924 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:18.494930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.495734 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.495738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:18.496326 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:18.496335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.496810 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.496813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:18.496900 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:18.497487 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:18.497499 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:18.499453 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
W20260808 10:35:18.499708 8980 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:18.494089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:35347 (local address 127.30.146.190:34763)
0808 10:35:18.494217 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:18.494224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:18.494241 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:35:18.494435 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:18.494439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:18.494496 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:18.494582 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:18.495053 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.495606 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.496453 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:18.496683 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:18.496801 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:18.499595 (+ 2794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:35347: BlockingRecv error: recv got EOF from 127.30.146.131:35347 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 10:35:19 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:19 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:19.119499 8984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:19.113066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:33099)
0808 10:35:19.113414 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:19.113427 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:19.113517 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:19.113918 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:19.113920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:19.114110 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:19.114207 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:19.114212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.115063 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.115066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:19.115640 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:19.115647 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.116122 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.116124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:19.116183 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:19.116873 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:19.116885 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:19.119377 (+ 2492us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":140,"threads_started":1}
W20260808 10:35:19.119680 8986 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:19.113244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:33099 (local address 127.30.146.190:34763)
0808 10:35:19.113670 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:19.113675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:19.113691 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:35:19.113734 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:19.113736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:19.113785 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:19.113880 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:19.114376 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.114959 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.115760 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.116019 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.116109 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:19.119560 (+ 3451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:33099: BlockingRecv error: recv got EOF from 127.30.146.129:33099 (error 108)
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":97,"threads_started":1}
Aug 08 10:35:19 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:19 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:19.311295 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:19.305283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:43179)
0808 10:35:19.305629 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:19.305645 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:19.305762 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:19.306020 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:19.306022 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:19.306263 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:19.306369 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:19.306374 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.307197 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.307200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:19.307802 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:19.307812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.308302 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.308305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:19.308378 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:19.308994 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:19.309008 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:19.311105 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":82,"threads_started":1}
W20260808 10:35:19.311401 8986 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:19.305454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:43179 (local address 127.30.146.190:34763)
0808 10:35:19.305637 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:19.305642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:19.305727 (+ 85us) server_negotiation.cc:407] Connection header received
0808 10:35:19.305849 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:19.305851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:19.305897 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:19.305982 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:19.306499 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.307056 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.307924 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.308182 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.308268 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:19.311270 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:43179: BlockingRecv error: recv got EOF from 127.30.146.130:43179 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 10:35:19 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:19 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.131@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:19.507040 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:19.500573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.131:52569)
0808 10:35:19.500948 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:19.500963 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:19.501065 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:19.501369 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:19.501371 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:19.501556 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:19.501654 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:19.501659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.502546 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.502549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:19.503155 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:19.503163 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.503752 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.503755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:19.503819 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:19.504387 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:19.504398 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:19.506901 (+ 2503us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":180,"threads_started":1}
W20260808 10:35:19.507225 8986 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:19.500615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:52569 (local address 127.30.146.190:34763)
0808 10:35:19.500793 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:19.500799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:19.501042 (+ 243us) server_negotiation.cc:407] Connection header received
0808 10:35:19.501202 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:19.501209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:19.501262 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:19.501328 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:19.501816 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.502432 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.503287 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:19.503547 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:19.503746 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:19.507080 (+ 3334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.131:52569: BlockingRecv error: recv got EOF from 127.30.146.131:52569 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 10:35:20 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:20 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.129@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:20.126904 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:20.120583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.129:40149)
0808 10:35:20.120901 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:20.120915 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:20.121020 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:20.121344 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:20.121347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:20.121559 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:20.121759 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:20.121766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.122546 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.122549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:20.123223 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:20.123233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.123720 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.123723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:20.123786 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:20.124367 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:20.124390 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:20.126760 (+ 2370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":104,"threads_started":1}
W20260808 10:35:20.127094 8990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:20.120716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:40149 (local address 127.30.146.190:34763)
0808 10:35:20.120973 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:20.120977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:20.120992 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:35:20.121118 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:20.121120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:20.121181 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:20.121261 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:20.121883 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.122417 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.123342 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.123574 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.123650 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:20.126920 (+ 3270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.129:40149: BlockingRecv error: recv got EOF from 127.30.146.129:40149 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":64,"threads_started":1}
Aug 08 10:35:20 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:35:20 dist-test-slave-9gcw krb5kdc[7799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.146.130@KRBTEST.COM for oryx/127.30.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:35:20.318591 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:20.312521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34763 (local address 127.30.146.130:46019)
0808 10:35:20.312859 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:20.312871 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:20.313004 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:20.313309 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:20.313312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:20.313533 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:20.313625 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:20.313631 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.314419 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.314426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:20.315023 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:20.315032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.315546 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.315549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:20.315635 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:20.316163 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:20.316174 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:20.318406 (+ 2232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.146.190:34763: Server oryx/127.30.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":138,"threads_started":1}
W20260808 10:35:20.318699 8990 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:20.312575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:46019 (local address 127.30.146.190:34763)
0808 10:35:20.312757 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:20.312762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:20.312950 (+ 188us) server_negotiation.cc:407] Connection header received
0808 10:35:20.313113 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:20.313116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:20.313192 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:20.313291 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:20.313751 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.314307 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.315145 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.315410 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.315540 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:20.318565 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.146.130:46019: BlockingRecv error: recv got EOF from 127.30.146.130:46019 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 10:35:20.388971 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 8451
I20260808 10:35:20.398137 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 8622
I20260808 10:35:20.406050 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 8776
I20260808 10:35:20.413020 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 8375
2026-08-08T10:35:20Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18261 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T10:35:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:35:20Z Disabled control of system clock
I20260808 10:35:20.448463 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:41817
--webserver_interface=127.30.146.190
--webserver_port=0
--builtin_ntp_servers=127.30.146.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:41817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 10:35:20.569808 8997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:20.570101 8997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:20.570138 8997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:20.574517 8997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:35:20.574589 8997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:20.574608 8997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:20.574631 8997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:35:20.574657 8997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:35:20.579663 8997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:41817
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:41817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:20.580866 8997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:20.582418 8997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:35:20.592289 8997 server_base.cc:1034] running on GCE node
W20260808 10:35:20.592195 9005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:20.592180 9003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:20.592195 9002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:20.593247 8997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:20.594349 8997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:20.595645 8997 hybrid_clock.cc:648] HybridClock initialized: now 1786185320595624 us; error 42 us; skew 500 ppm
I20260808 10:35:20.597581 8997 webserver.cc:460] Webserver started at http://127.30.146.190:35257/ using document root <none> and password file <none>
I20260808 10:35:20.598146 8997 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:20.598207 8997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:20.598380 8997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:20.600461 8997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "c320ca106020471c91592d739d1aa90b"
format_stamp: "Formatted at 2026-08-08 10:35:20 on dist-test-slave-9gcw"
server_key: "cbddd386afed91b6e217df056f213b1e"
server_key_iv: "f0e724c2a957eb3ba2da8c40ff34b057"
server_key_version: "encryptionkey@0"
I20260808 10:35:20.600983 8997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "c320ca106020471c91592d739d1aa90b"
format_stamp: "Formatted at 2026-08-08 10:35:20 on dist-test-slave-9gcw"
server_key: "cbddd386afed91b6e217df056f213b1e"
server_key_iv: "f0e724c2a957eb3ba2da8c40ff34b057"
server_key_version: "encryptionkey@0"
I20260808 10:35:20.604818 8997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:35:20.607504 9012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:20.608981 8997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:35:20.609241 8997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "c320ca106020471c91592d739d1aa90b"
format_stamp: "Formatted at 2026-08-08 10:35:20 on dist-test-slave-9gcw"
server_key: "cbddd386afed91b6e217df056f213b1e"
server_key_iv: "f0e724c2a957eb3ba2da8c40ff34b057"
server_key_version: "encryptionkey@0"
I20260808 10:35:20.609385 8997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:20.636217 8997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:20.637089 8997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:20.637351 8997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:20.645916 8997 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:41817
I20260808 10:35:20.645917 9069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:41817 every 8 connection(s)
I20260808 10:35:20.647204 8997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:35:20.650682 9070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:20.655766 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 8997
I20260808 10:35:20.655917 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:35:20.656242 31306 external_mini_cluster.cc:1428] Setting key e1f7f9ac85c7bb9cc83df52f450b1134
I20260808 10:35:20.656875 9070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b: Bootstrap starting.
I20260808 10:35:20.659605 9070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:20.660590 9070 log.cc:824] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:35:20.662933 9070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b: No bootstrap required, opened a new log
I20260808 10:35:20.667555 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:20.657813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39352 (local address 127.30.146.190:41817)
0808 10:35:20.658527 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:20.658546 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:20.658591 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:35:20.659405 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:20.659430 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:20.659962 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:20.660366 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:20.660970 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.662362 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.663426 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.664155 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.664327 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:20.665234 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:20.665248 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:20.665267 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:35:20.665775 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:20.665848 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:35:20.665874 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:20.665976 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:20.666126 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 10:35:20.666465 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":122,"threads_started":1}
I20260808 10:35:20.667586 9070 raft_consensus.cc:348] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320ca106020471c91592d739d1aa90b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41817 } }
I20260808 10:35:20.667858 9070 raft_consensus.cc:374] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:20.667928 9070 raft_consensus.cc:729] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c320ca106020471c91592d739d1aa90b, State: Initialized, Role: FOLLOWER
I20260808 10:35:20.668397 9070 consensus_queue.cc:260] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320ca106020471c91592d739d1aa90b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41817 } }
I20260808 10:35:20.668560 9070 raft_consensus.cc:388] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:35:20.668663 9070 raft_consensus.cc:482] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:35:20.668797 9070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:20.669836 9070 raft_consensus.cc:504] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320ca106020471c91592d739d1aa90b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41817 } }
I20260808 10:35:20.670176 9070 leader_election.cc:302] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c320ca106020471c91592d739d1aa90b; no voters:
I20260808 10:35:20.670435 9070 leader_election.cc:288] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:35:20.670572 9075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:35:20.670918 9075 raft_consensus.cc:686] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [term 1 LEADER]: Becoming Leader. State: Replica: c320ca106020471c91592d739d1aa90b, State: Running, Role: LEADER
I20260808 10:35:20.671262 9075 consensus_queue.cc:237] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320ca106020471c91592d739d1aa90b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41817 } }
I20260808 10:35:20.671679 9070 sys_catalog.cc:564] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:35:20.673070 9076 sys_catalog.cc:455] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c320ca106020471c91592d739d1aa90b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320ca106020471c91592d739d1aa90b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41817 } } }
I20260808 10:35:20.673098 9077 sys_catalog.cc:455] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c320ca106020471c91592d739d1aa90b. Latest consensus state: current_term: 1 leader_uuid: "c320ca106020471c91592d739d1aa90b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320ca106020471c91592d739d1aa90b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41817 } } }
I20260808 10:35:20.673275 9077 sys_catalog.cc:458] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [sys.catalog]: This master's current role is: LEADER
I20260808 10:35:20.673264 9076 sys_catalog.cc:458] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b [sys.catalog]: This master's current role is: LEADER
I20260808 10:35:20.673916 9086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:35:20.677098 9086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:35:20.682898 9086 catalog_manager.cc:1347] Generated new cluster ID: 43aed86381b64986b45198614790be3a
I20260808 10:35:20.682986 9086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:35:20.701565 9086 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:35:20.702827 9086 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:35:20.707912 9086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c320ca106020471c91592d739d1aa90b: Generated new TSK 0
I20260808 10:35:20.708699 9086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:35:20.713774 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41817
--builtin_ntp_servers=127.30.146.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 10:35:20.838712 9096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:20.838984 9096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:20.839030 9096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:20.843276 9096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:20.843338 9096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:20.843391 9096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:35:20.848198 9096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:20.849380 9096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:20.850821 9096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:20.861004 9101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:20.861291 9102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:20.861229 9096 server_base.cc:1034] running on GCE node
W20260808 10:35:20.861439 9104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:20.861855 9096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:20.862529 9096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:20.863730 9096 hybrid_clock.cc:648] HybridClock initialized: now 1786185320863706 us; error 59 us; skew 500 ppm
I20260808 10:35:20.865845 9096 webserver.cc:460] Webserver started at http://127.30.146.129:44069/ using document root <none> and password file <none>
I20260808 10:35:20.866465 9096 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:20.866564 9096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:20.866837 9096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:20.868942 9096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "384a30d8f00f47da8e93e09902857b03"
format_stamp: "Formatted at 2026-08-08 10:35:20 on dist-test-slave-9gcw"
server_key: "29fa5b23f8ba8be7e3577bcac9d7fbd7"
server_key_iv: "92406723668cbf91cde5eb681c132f08"
server_key_version: "encryptionkey@0"
I20260808 10:35:20.869575 9096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "384a30d8f00f47da8e93e09902857b03"
format_stamp: "Formatted at 2026-08-08 10:35:20 on dist-test-slave-9gcw"
server_key: "29fa5b23f8ba8be7e3577bcac9d7fbd7"
server_key_iv: "92406723668cbf91cde5eb681c132f08"
server_key_version: "encryptionkey@0"
I20260808 10:35:20.873770 9096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:35:20.876361 9110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:20.877661 9096 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:35:20.877826 9096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "384a30d8f00f47da8e93e09902857b03"
format_stamp: "Formatted at 2026-08-08 10:35:20 on dist-test-slave-9gcw"
server_key: "29fa5b23f8ba8be7e3577bcac9d7fbd7"
server_key_iv: "92406723668cbf91cde5eb681c132f08"
server_key_version: "encryptionkey@0"
I20260808 10:35:20.877948 9096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:20.901939 9096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:20.902675 9096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:20.902892 9096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:20.903528 9096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:20.904534 9096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:20.904608 9096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:20.904680 9096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:20.904731 9096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:20.914237 9096 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:42295
I20260808 10:35:20.914264 9236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:42295 every 8 connection(s)
I20260808 10:35:20.915222 9096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:35:20.922133 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 9096
I20260808 10:35:20.922302 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:35:20.922675 31306 external_mini_cluster.cc:1428] Setting key 03d07109d290a1cdc97d51e0e3fdd1fd
I20260808 10:35:20.924891 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41817
--builtin_ntp_servers=127.30.146.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 10:35:20.927402 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:20.917243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:60689 (local address 127.30.146.190:41817)
0808 10:35:20.917391 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:20.917394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:20.918527 (+ 1133us) server_negotiation.cc:407] Connection header received
0808 10:35:20.919979 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:20.919982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:20.920043 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:20.920154 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:20.921070 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.921852 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.923626 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.924068 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.924196 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:20.926573 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:20.926601 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:20.926606 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:35:20.926624 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:20.926652 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:35:20.926670 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:20.926757 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:20.926975 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 10:35:20.927085 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:35:20.928488 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:20.917655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:41817 (local address 127.30.146.129:60689)
0808 10:35:20.918338 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:20.918379 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:20.919759 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:20.920347 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:20.920354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:20.920436 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:20.920922 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:20.920934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.922011 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.922015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:20.923486 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:20.923497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:20.924279 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:20.924284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:20.925438 (+ 1154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:20.926196 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:35:20.926200 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:20.926266 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:35:20.926360 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:20.926777 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:20.926791 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:20.926816 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:35:20.926959 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 10:35:20.927310 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":117,"threads_started":1}
I20260808 10:35:20.929901 9237 heartbeater.cc:347] Connected to a master server at 127.30.146.190:41817
I20260808 10:35:20.930172 9237 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:20.930887 9237 heartbeater.cc:511] Master 127.30.146.190:41817 requested a full tablet report, sending...
I20260808 10:35:20.932834 9032 ts_manager.cc:194] Registered new tserver with Master: 384a30d8f00f47da8e93e09902857b03 (127.30.146.129:42295)
I20260808 10:35:20.934656 9032 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.129:60689
W20260808 10:35:21.045338 9240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:21.045624 9240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:21.045662 9240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:21.049721 9240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:21.049775 9240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:21.049829 9240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:35:21.054661 9240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:21.055760 9240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:21.057381 9240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:21.067564 9248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:21.067638 9240 server_base.cc:1034] running on GCE node
W20260808 10:35:21.067700 9245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:21.067776 9246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:21.068161 9240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:21.068761 9240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:21.069931 9240 hybrid_clock.cc:648] HybridClock initialized: now 1786185321069919 us; error 24 us; skew 500 ppm
I20260808 10:35:21.071866 9240 webserver.cc:460] Webserver started at http://127.30.146.130:46633/ using document root <none> and password file <none>
I20260808 10:35:21.072515 9240 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:21.072583 9240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:21.072888 9240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:21.075019 9240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "4ac95d44312b4a2f9c1c0b181370730f"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "6d8d5227fefb2da3709fea1d172b9a65"
server_key_iv: "33cb21150425bf298e32874927c2540c"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.075603 9240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "4ac95d44312b4a2f9c1c0b181370730f"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "6d8d5227fefb2da3709fea1d172b9a65"
server_key_iv: "33cb21150425bf298e32874927c2540c"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.079311 9240 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:35:21.082177 9255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.083446 9240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:35:21.083540 9240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "4ac95d44312b4a2f9c1c0b181370730f"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "6d8d5227fefb2da3709fea1d172b9a65"
server_key_iv: "33cb21150425bf298e32874927c2540c"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.083629 9240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:21.097676 9240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:21.098325 9240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:21.098475 9240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:21.098995 9240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:21.099918 9240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:21.099967 9240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.100006 9240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:21.100021 9240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.108776 9240 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:45079
I20260808 10:35:21.108851 9379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:45079 every 8 connection(s)
I20260808 10:35:21.109874 9240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:35:21.114603 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 9240
I20260808 10:35:21.114823 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:35:21.115131 31306 external_mini_cluster.cc:1428] Setting key 47a7780dd4d107895ab5c0373d01b04f
I20260808 10:35:21.117240 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41817
--builtin_ntp_servers=127.30.146.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 10:35:21.123406 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:21.111704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:60025 (local address 127.30.146.190:41817)
0808 10:35:21.111854 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:21.111858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:21.112841 (+ 983us) server_negotiation.cc:407] Connection header received
0808 10:35:21.114390 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:21.114393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:21.114455 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:21.114556 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:21.115610 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:21.116310 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:21.117688 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:21.117911 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:21.118024 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:21.122798 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:21.122802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:21.122804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:35:21.122820 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:21.122841 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:35:21.122851 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:21.122940 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:21.123172 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 10:35:21.123245 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:35:21.124394 9382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:21.112051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:41817 (local address 127.30.146.130:60025)
0808 10:35:21.112674 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:21.112715 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:21.114183 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:21.114721 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:21.114754 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:21.114866 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:21.115442 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:21.115457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:21.116446 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:21.116450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:21.117546 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:21.117556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:21.118069 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:21.118073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:21.118249 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:21.122325 (+ 4076us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:35:21.122327 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:21.122373 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:35:21.122448 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:21.122957 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:21.122971 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:21.122978 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:35:21.123154 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 10:35:21.123486 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":108,"threads_started":1}
I20260808 10:35:21.125901 9380 heartbeater.cc:347] Connected to a master server at 127.30.146.190:41817
I20260808 10:35:21.126173 9380 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:21.126713 9380 heartbeater.cc:511] Master 127.30.146.190:41817 requested a full tablet report, sending...
I20260808 10:35:21.127803 9032 ts_manager.cc:194] Registered new tserver with Master: 4ac95d44312b4a2f9c1c0b181370730f (127.30.146.130:45079)
I20260808 10:35:21.128372 9032 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.130:60025
W20260808 10:35:21.239348 9383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:21.239651 9383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:21.239729 9383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:21.244350 9383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:21.244444 9383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:21.244547 9383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:35:21.249629 9383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:41817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:21.250784 9383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:21.252166 9383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:21.262089 9391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:21.262065 9389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:21.262269 9383 server_base.cc:1034] running on GCE node
W20260808 10:35:21.262454 9388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:21.262877 9383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:21.263602 9383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:21.264809 9383 hybrid_clock.cc:648] HybridClock initialized: now 1786185321264780 us; error 50 us; skew 500 ppm
I20260808 10:35:21.266706 9383 webserver.cc:460] Webserver started at http://127.30.146.131:44123/ using document root <none> and password file <none>
I20260808 10:35:21.267309 9383 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:21.267406 9383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:21.267647 9383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:21.269847 9383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "b55d179454a945c0a812808c7be5ae77"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "5af0784fad180a5fe1bf68e418e2e0fd"
server_key_iv: "4e488863bf269d1884c4e350abca5fd0"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.270423 9383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "b55d179454a945c0a812808c7be5ae77"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "5af0784fad180a5fe1bf68e418e2e0fd"
server_key_iv: "4e488863bf269d1884c4e350abca5fd0"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.274441 9383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:35:21.277007 9403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.278270 9383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:35:21.278420 9383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "b55d179454a945c0a812808c7be5ae77"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "5af0784fad180a5fe1bf68e418e2e0fd"
server_key_iv: "4e488863bf269d1884c4e350abca5fd0"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.278537 9383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:21.290632 9383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:21.291321 9383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:21.291522 9383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:21.292116 9383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:21.293053 9383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:21.293123 9383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.293228 9383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:21.293277 9383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.302295 9383 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:44121
I20260808 10:35:21.302330 9530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:44121 every 8 connection(s)
I20260808 10:35:21.303630 9383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:35:21.304435 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 9383
I20260808 10:35:21.304540 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:35:21.304840 31306 external_mini_cluster.cc:1428] Setting key 70da526587322075cb9542ce32c8cad7
I20260808 10:35:21.313484 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:21.305812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:44137 (local address 127.30.146.190:41817)
0808 10:35:21.305944 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:21.305952 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:21.307305 (+ 1353us) server_negotiation.cc:407] Connection header received
0808 10:35:21.308695 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:21.308698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:21.308746 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:21.308815 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:21.309722 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:21.310320 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:21.311488 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:21.311818 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:21.311922 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:21.312972 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:21.312975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:21.312977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:35:21.312989 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:21.313009 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:35:21.313017 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:21.313083 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:21.313308 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 10:35:21.313363 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:35:21.314441 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:21.306297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:41817 (local address 127.30.146.131:44137)
0808 10:35:21.307142 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:21.307197 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:21.308498 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:21.308947 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:21.308953 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:21.309032 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:21.309586 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:21.309599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:21.310435 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:21.310438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:21.311368 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:21.311377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:21.311967 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:21.311970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:21.312105 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:21.312715 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:35:21.312716 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:21.312761 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:35:21.312822 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:21.313107 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:21.313118 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:21.313125 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:35:21.313271 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 10:35:21.313551 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":167,"threads_started":1}
I20260808 10:35:21.315641 9531 heartbeater.cc:347] Connected to a master server at 127.30.146.190:41817
I20260808 10:35:21.315877 9531 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:21.316402 9531 heartbeater.cc:511] Master 127.30.146.190:41817 requested a full tablet report, sending...
I20260808 10:35:21.317474 9032 ts_manager.cc:194] Registered new tserver with Master: b55d179454a945c0a812808c7be5ae77 (127.30.146.131:44121)
I20260808 10:35:21.318152 9032 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.131:44137
I20260808 10:35:21.319348 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:35:21.322649 9073 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:21.321727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39360 (local address 127.30.146.190:41817)
0808 10:35:21.321853 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:21.321856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:21.322038 (+ 182us) server_negotiation.cc:407] Connection header received
0808 10:35:21.322182 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:21.322185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:21.322222 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:21.322294 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:21.322550 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39360: BlockingRecv error: recv got EOF from 127.0.0.1:39360 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:35:21.322889 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 9096
I20260808 10:35:21.332186 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 9240
I20260808 10:35:21.341409 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 9383
I20260808 10:35:21.348938 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 8997
2026-08-08T10:35:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (933 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T10:35:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:35:21Z Disabled control of system clock
I20260808 10:35:21.381896 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:37215
--webserver_interface=127.30.146.190
--webserver_port=0
--builtin_ntp_servers=127.30.146.148:33141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:37215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:35:21.496767 9544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:21.497185 9544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:21.497243 9544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:21.501356 9544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:35:21.501425 9544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:21.501441 9544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:21.501457 9544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:35:21.501472 9544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:35:21.506287 9544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:37215
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:37215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:21.507434 9544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:21.508805 9544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:21.517441 9550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:21.517460 9549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:21.517575 9552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:21.517897 9544 server_base.cc:1034] running on GCE node
I20260808 10:35:21.518462 9544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:21.519690 9544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:21.520872 9544 hybrid_clock.cc:648] HybridClock initialized: now 1786185321520839 us; error 43 us; skew 500 ppm
I20260808 10:35:21.523133 9544 webserver.cc:460] Webserver started at http://127.30.146.190:36257/ using document root <none> and password file <none>
I20260808 10:35:21.523768 9544 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:21.523838 9544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:21.524122 9544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:21.526443 9544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "a0898b28cc6c47448d098b5f099fde04"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "89425b7348ac3bedf751fc9858ef701d"
server_key_iv: "8fed287cefcd7476ddaf615834a39dcb"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.527055 9544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "a0898b28cc6c47448d098b5f099fde04"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "89425b7348ac3bedf751fc9858ef701d"
server_key_iv: "8fed287cefcd7476ddaf615834a39dcb"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.531198 9544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 10:35:21.533977 9561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.535440 9544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:35:21.535615 9544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "a0898b28cc6c47448d098b5f099fde04"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "89425b7348ac3bedf751fc9858ef701d"
server_key_iv: "8fed287cefcd7476ddaf615834a39dcb"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.535756 9544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:21.540596 9544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:21.541296 9544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:21.541487 9544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:21.548914 9544 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:37215
I20260808 10:35:21.548925 9616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:37215 every 8 connection(s)
I20260808 10:35:21.549911 9544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:35:21.553390 9617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:21.558145 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 9544
I20260808 10:35:21.558290 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:35:21.558681 31306 external_mini_cluster.cc:1428] Setting key a3687159628611c7dd7bd6b272c55a37
I20260808 10:35:21.559755 9617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04: Bootstrap starting.
I20260808 10:35:21.562289 9617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:21.563251 9617 log.cc:824] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04: Log is configured to *not* fsync() on all Append() calls
I20260808 10:35:21.565044 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:21.560327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32868 (local address 127.30.146.190:37215)
0808 10:35:21.560986 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:21.561003 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:21.561039 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:35:21.561779 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:21.561797 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:21.562170 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:21.562534 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:21.563486 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:21.563498 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:21.563510 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:35:21.563519 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:21.563571 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:35:21.563587 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:21.563712 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:21.563838 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 10:35:21.564130 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":166,"threads_started":1}
I20260808 10:35:21.565692 9617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04: No bootstrap required, opened a new log
I20260808 10:35:21.569526 9617 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0898b28cc6c47448d098b5f099fde04" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37215 } }
I20260808 10:35:21.569775 9617 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:21.569819 9617 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0898b28cc6c47448d098b5f099fde04, State: Initialized, Role: FOLLOWER
I20260808 10:35:21.570238 9617 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0898b28cc6c47448d098b5f099fde04" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37215 } }
I20260808 10:35:21.570364 9617 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:35:21.570411 9617 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:35:21.570484 9617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:21.571614 9617 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0898b28cc6c47448d098b5f099fde04" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37215 } }
I20260808 10:35:21.571919 9617 leader_election.cc:302] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0898b28cc6c47448d098b5f099fde04; no voters:
I20260808 10:35:21.572190 9617 leader_election.cc:288] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:35:21.572330 9622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:35:21.572599 9622 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [term 1 LEADER]: Becoming Leader. State: Replica: a0898b28cc6c47448d098b5f099fde04, State: Running, Role: LEADER
I20260808 10:35:21.572907 9622 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0898b28cc6c47448d098b5f099fde04" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37215 } }
I20260808 10:35:21.573587 9617 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:35:21.574891 9624 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0898b28cc6c47448d098b5f099fde04. Latest consensus state: current_term: 1 leader_uuid: "a0898b28cc6c47448d098b5f099fde04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0898b28cc6c47448d098b5f099fde04" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37215 } } }
I20260808 10:35:21.575040 9624 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [sys.catalog]: This master's current role is: LEADER
I20260808 10:35:21.574896 9623 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0898b28cc6c47448d098b5f099fde04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0898b28cc6c47448d098b5f099fde04" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37215 } } }
I20260808 10:35:21.575424 9623 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04 [sys.catalog]: This master's current role is: LEADER
I20260808 10:35:21.575894 9634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:35:21.578677 9634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:35:21.584162 9634 catalog_manager.cc:1347] Generated new cluster ID: 60a1017502604957b02a35a5d4c81468
I20260808 10:35:21.584244 9634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:35:21.593282 9634 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:35:21.593649 9634 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:35:21.603633 9634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0898b28cc6c47448d098b5f099fde04: Generated new TSK 0
I20260808 10:35:21.604413 9634 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:35:21.619017 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37215
--builtin_ntp_servers=127.30.146.148:33141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:35:21.739306 9644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:21.739586 9644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:21.739657 9644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:21.743757 9644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:21.743824 9644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:21.743880 9644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:35:21.749013 9644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:21.750233 9644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:21.751660 9644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:21.762276 9652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:21.762297 9650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:21.762276 9649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:21.762693 9644 server_base.cc:1034] running on GCE node
I20260808 10:35:21.763343 9644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:21.764101 9644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:21.765305 9644 hybrid_clock.cc:648] HybridClock initialized: now 1786185321765292 us; error 29 us; skew 500 ppm
I20260808 10:35:21.767362 9644 webserver.cc:460] Webserver started at http://127.30.146.129:46199/ using document root <none> and password file <none>
I20260808 10:35:21.768009 9644 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:21.768111 9644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:21.768380 9644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:21.770798 9644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "25b98d58c75a482891c20b471ad424d6"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "100b7868d95e47d2bf3e516b1ee41502"
server_key_iv: "29de73aa31f39bc9e7f572189b5279ca"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.771425 9644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "25b98d58c75a482891c20b471ad424d6"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "100b7868d95e47d2bf3e516b1ee41502"
server_key_iv: "29de73aa31f39bc9e7f572189b5279ca"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.775790 9644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:35:21.778595 9659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.779874 9644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:35:21.779992 9644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "25b98d58c75a482891c20b471ad424d6"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "100b7868d95e47d2bf3e516b1ee41502"
server_key_iv: "29de73aa31f39bc9e7f572189b5279ca"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.780113 9644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:21.784893 9644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:21.785833 9644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:21.786068 9644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:21.786738 9644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:21.787802 9644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:21.787856 9644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.787931 9644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:21.787964 9644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.797755 9644 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:38159
I20260808 10:35:21.797782 9782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:38159 every 8 connection(s)
I20260808 10:35:21.798781 9644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:35:21.805344 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:21.800702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:34743 (local address 127.30.146.190:37215)
0808 10:35:21.800853 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:21.800857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:21.802074 (+ 1217us) server_negotiation.cc:407] Connection header received
0808 10:35:21.803530 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:21.803533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:21.803580 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:21.803694 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:21.804843 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:21.804847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:21.804850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:35:21.804861 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:21.804887 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:35:21.804895 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:21.804994 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:21.805134 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 10:35:21.805197 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:35:21.806578 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 9644
I20260808 10:35:21.806524 9785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:21.801136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:37215 (local address 127.30.146.129:34743)
0808 10:35:21.801919 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:21.801962 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:21.803305 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:21.803871 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:21.803878 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:21.803969 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:21.804584 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:35:21.804586 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:21.804625 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:35:21.804692 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:21.804976 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:21.804986 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:21.804992 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:35:21.805198 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 10:35:21.805479 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":116,"threads_started":1}
I20260808 10:35:21.806746 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:35:21.807163 31306 external_mini_cluster.cc:1428] Setting key 3a215242f3746df895147b4134ce3f28
I20260808 10:35:21.808143 9783 heartbeater.cc:347] Connected to a master server at 127.30.146.190:37215
I20260808 10:35:21.808403 9783 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:21.808930 9783 heartbeater.cc:511] Master 127.30.146.190:37215 requested a full tablet report, sending...
I20260808 10:35:21.809554 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37215
--builtin_ntp_servers=127.30.146.148:33141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 10:35:21.810591 9579 ts_manager.cc:194] Registered new tserver with Master: 25b98d58c75a482891c20b471ad424d6 (127.30.146.129:38159)
W20260808 10:35:21.924964 9787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:21.925354 9787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:21.925426 9787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:21.929412 9787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:21.929459 9787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:21.929546 9787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:35:21.934422 9787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:21.935500 9787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:21.936874 9787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:21.947109 9795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:21.947147 9792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:21.947124 9793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:21.947451 9787 server_base.cc:1034] running on GCE node
I20260808 10:35:21.947875 9787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:21.948589 9787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:21.949767 9787 hybrid_clock.cc:648] HybridClock initialized: now 1786185321949731 us; error 51 us; skew 500 ppm
I20260808 10:35:21.951838 9787 webserver.cc:460] Webserver started at http://127.30.146.130:46881/ using document root <none> and password file <none>
I20260808 10:35:21.952437 9787 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:21.952502 9787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:21.952729 9787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:21.954882 9787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "16a77d10d84b4cd181cabe7c4e219e8c"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "cd94ce67bc651f545f6adc600c913e64"
server_key_iv: "ac68b1a9e6c4d34a7cf4c0d8667dd041"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.955487 9787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "16a77d10d84b4cd181cabe7c4e219e8c"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "cd94ce67bc651f545f6adc600c913e64"
server_key_iv: "ac68b1a9e6c4d34a7cf4c0d8667dd041"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.959518 9787 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 10:35:21.962311 9801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.963724 9787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:35:21.963860 9787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "16a77d10d84b4cd181cabe7c4e219e8c"
format_stamp: "Formatted at 2026-08-08 10:35:21 on dist-test-slave-9gcw"
server_key: "cd94ce67bc651f545f6adc600c913e64"
server_key_iv: "ac68b1a9e6c4d34a7cf4c0d8667dd041"
server_key_version: "encryptionkey@0"
I20260808 10:35:21.963980 9787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:21.968863 9787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:21.969539 9787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:21.969734 9787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:21.970294 9787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:21.971203 9787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:21.971251 9787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.971315 9787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:21.971349 9787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:21.981120 9787 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:40109
I20260808 10:35:21.981220 9924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:40109 every 8 connection(s)
I20260808 10:35:21.982095 9787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:35:21.987529 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 9787
I20260808 10:35:21.987677 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:35:21.988045 31306 external_mini_cluster.cc:1428] Setting key e7bee44d964f357e7540f64a26bb144e
I20260808 10:35:21.989081 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:21.984361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:59145 (local address 127.30.146.190:37215)
0808 10:35:21.984530 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:21.984535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:21.985647 (+ 1112us) server_negotiation.cc:407] Connection header received
0808 10:35:21.987009 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:21.987012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:21.987049 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:21.987134 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:21.988510 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:21.988514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:21.988518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:35:21.988532 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:21.988555 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:35:21.988565 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:21.988671 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:21.988887 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 10:35:21.988953 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:35:21.990259 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:21.984786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:37215 (local address 127.30.146.130:59145)
0808 10:35:21.985493 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:21.985533 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:21.986810 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:21.987353 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:21.987361 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:21.987483 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:21.988183 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:35:21.988185 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:21.988239 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:35:21.988327 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:21.988725 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:21.988735 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:21.988742 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:35:21.988922 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 10:35:21.989132 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":119,"threads_started":1}
I20260808 10:35:21.990401 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37215
--builtin_ntp_servers=127.30.146.148:33141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 10:35:21.991449 9925 heartbeater.cc:347] Connected to a master server at 127.30.146.190:37215
I20260808 10:35:21.991701 9925 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:21.992106 9925 heartbeater.cc:511] Master 127.30.146.190:37215 requested a full tablet report, sending...
I20260808 10:35:21.993361 9579 ts_manager.cc:194] Registered new tserver with Master: 16a77d10d84b4cd181cabe7c4e219e8c (127.30.146.130:40109)
W20260808 10:35:22.108414 9928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:22.108737 9928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:22.108801 9928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:22.112885 9928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:22.112949 9928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:22.113008 9928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:35:22.118314 9928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:22.119608 9928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:22.121042 9928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:22.131194 9933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:22.131126 9934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:22.131137 9936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:22.131628 9928 server_base.cc:1034] running on GCE node
I20260808 10:35:22.132081 9928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:22.132766 9928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:22.133993 9928 hybrid_clock.cc:648] HybridClock initialized: now 1786185322133988 us; error 34 us; skew 500 ppm
I20260808 10:35:22.136032 9928 webserver.cc:460] Webserver started at http://127.30.146.131:39449/ using document root <none> and password file <none>
I20260808 10:35:22.136633 9928 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:22.136734 9928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:22.136972 9928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:22.139341 9928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "8f8d4db0843943a497bf6f545b45d9bc"
format_stamp: "Formatted at 2026-08-08 10:35:22 on dist-test-slave-9gcw"
server_key: "2ea89ff93e019fe51d0262d76111d374"
server_key_iv: "e8ac5efe2daca1610db2ded38ecf3b1c"
server_key_version: "encryptionkey@0"
I20260808 10:35:22.140009 9928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "8f8d4db0843943a497bf6f545b45d9bc"
format_stamp: "Formatted at 2026-08-08 10:35:22 on dist-test-slave-9gcw"
server_key: "2ea89ff93e019fe51d0262d76111d374"
server_key_iv: "e8ac5efe2daca1610db2ded38ecf3b1c"
server_key_version: "encryptionkey@0"
I20260808 10:35:22.143898 9928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 10:35:22.147006 9942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:22.148485 9928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:35:22.148630 9928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "8f8d4db0843943a497bf6f545b45d9bc"
format_stamp: "Formatted at 2026-08-08 10:35:22 on dist-test-slave-9gcw"
server_key: "2ea89ff93e019fe51d0262d76111d374"
server_key_iv: "e8ac5efe2daca1610db2ded38ecf3b1c"
server_key_version: "encryptionkey@0"
I20260808 10:35:22.148759 9928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:22.153615 9928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:22.154265 9928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:22.154457 9928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:22.155043 9928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:22.156069 9928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:22.156150 9928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:22.156224 9928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:22.156281 9928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:22.165808 9928 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:34713
I20260808 10:35:22.165828 10066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:34713 every 8 connection(s)
I20260808 10:35:22.166800 9928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:35:22.168744 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 9928
I20260808 10:35:22.168883 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:35:22.169286 31306 external_mini_cluster.cc:1428] Setting key 0482b5d3142bb5cf372848fd4b3bf95e
I20260808 10:35:22.174237 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:22.169066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:60973 (local address 127.30.146.190:37215)
0808 10:35:22.169275 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:22.169280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:22.170672 (+ 1392us) server_negotiation.cc:407] Connection header received
0808 10:35:22.172214 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:22.172217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:22.172263 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:22.172341 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:22.173670 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:22.173674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:22.173677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:35:22.173690 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:22.173713 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:35:22.173729 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:22.173831 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:22.174032 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 10:35:22.174084 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 10:35:22.175333 10069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:22.169609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:37215 (local address 127.30.146.131:60973)
0808 10:35:22.170447 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:22.170532 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:22.171983 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:22.172521 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:22.172528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:22.172655 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:22.173398 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:35:22.173400 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:22.173443 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:35:22.173514 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:22.173817 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:22.173826 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:22.173833 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:35:22.174084 (+ 251us) client_negotiation.cc:240] Negotiation successful
0808 10:35:22.174342 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":160,"threads_started":1}
I20260808 10:35:22.176489 10067 heartbeater.cc:347] Connected to a master server at 127.30.146.190:37215
I20260808 10:35:22.176766 10067 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:22.177297 10067 heartbeater.cc:511] Master 127.30.146.190:37215 requested a full tablet report, sending...
I20260808 10:35:22.178427 9579 ts_manager.cc:194] Registered new tserver with Master: 8f8d4db0843943a497bf6f545b45d9bc (127.30.146.131:34713)
I20260808 10:35:22.183765 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:35:22.187458 9620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:35:22.186261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32882 (local address 127.30.146.190:37215)
0808 10:35:22.186415 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:22.186419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:22.186703 (+ 284us) server_negotiation.cc:407] Connection header received
0808 10:35:22.186808 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:22.186810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:22.186850 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:22.186924 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:22.187278 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32882: BlockingRecv error: recv got EOF from 127.0.0.1:32882 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:35:22.187943 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 9644
I20260808 10:35:22.198130 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 9787
I20260808 10:35:22.206846 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 9928
I20260808 10:35:22.213886 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 9544
2026-08-08T10:35:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (864 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:35:22 dist-test-slave-9gcw krb5kdc[10078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:35:22 dist-test-slave-9gcw krb5kdc[10078](info): set up 2 sockets
Aug 08 10:35:22 dist-test-slave-9gcw krb5kdc[10078](info): commencing operation
krb5kdc: starting...
W20260808 10:35:23.322469 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:35:23 dist-test-slave-9gcw krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:35:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:35:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:35:23.461702 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:42547
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:42547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:35:23.576464 10092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:23.576750 10092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:23.576792 10092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:23.580971 10092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:35:23.581035 10092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:23.581054 10092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:23.581071 10092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:35:23.581117 10092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:35:23.586648 10092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:42547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:42547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:23.587719 10092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:23.588812 10092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:23.597368 10097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:23.597491 10100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:23.597600 10092 server_base.cc:1034] running on GCE node
W20260808 10:35:23.597491 10098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:23.598305 10092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:23.599469 10092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:23.600646 10092 hybrid_clock.cc:648] HybridClock initialized: now 1786185323600622 us; error 37 us; skew 500 ppm
Aug 08 10:35:23 dist-test-slave-9gcw krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185323, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:23.603564 10092 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:35:23.604437 10092 webserver.cc:460] Webserver started at http://127.30.146.190:44321/ using document root <none> and password file <none>
I20260808 10:35:23.605010 10092 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:23.605096 10092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:23.605379 10092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:23.607558 10092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "48dbda3315fe4997b3f36395a6f7bdfb"
format_stamp: "Formatted at 2026-08-08 10:35:23 on dist-test-slave-9gcw"
server_key: "5b2099e2330d5cde808eff20df9be3eb"
server_key_iv: "770af23d3b9b5205e7740d3a5bc619dc"
server_key_version: "encryptionkey@0"
I20260808 10:35:23.608080 10092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "48dbda3315fe4997b3f36395a6f7bdfb"
format_stamp: "Formatted at 2026-08-08 10:35:23 on dist-test-slave-9gcw"
server_key: "5b2099e2330d5cde808eff20df9be3eb"
server_key_iv: "770af23d3b9b5205e7740d3a5bc619dc"
server_key_version: "encryptionkey@0"
I20260808 10:35:23.611847 10092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 10:35:23.614434 10107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:23.615752 10092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:35:23.615899 10092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "48dbda3315fe4997b3f36395a6f7bdfb"
format_stamp: "Formatted at 2026-08-08 10:35:23 on dist-test-slave-9gcw"
server_key: "5b2099e2330d5cde808eff20df9be3eb"
server_key_iv: "770af23d3b9b5205e7740d3a5bc619dc"
server_key_version: "encryptionkey@0"
I20260808 10:35:23.616009 10092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:23.642171 10092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:23.645956 10092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:23.646185 10092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:23.653684 10092 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:42547
I20260808 10:35:23.653985 10164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:42547 every 8 connection(s)
I20260808 10:35:23.654795 10092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:35:23.658507 10166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:23.659582 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 10092
I20260808 10:35:23.659755 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:35:23.660068 31306 external_mini_cluster.cc:1428] Setting key 710ab3c8192776f4aaa4d50af5b1c9c1
I20260808 10:35:23.664651 10166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb: Bootstrap starting.
I20260808 10:35:23.667061 10166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:23.667943 10166 log.cc:824] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:35:23 dist-test-slave-9gcw krb5kdc[10078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:35:23.670085 10166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb: No bootstrap required, opened a new log
I20260808 10:35:23.673678 10166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dbda3315fe4997b3f36395a6f7bdfb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42547 } }
I20260808 10:35:23.673933 10166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:23.674014 10166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dbda3315fe4997b3f36395a6f7bdfb, State: Initialized, Role: FOLLOWER
I20260808 10:35:23.674396 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:23.661559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45378 (local address 127.30.146.190:42547)
0808 10:35:23.661995 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:23.662011 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:23.662047 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:35:23.662688 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:23.662704 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:23.663029 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:23.663369 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:23.664246 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:23.665344 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:23.666204 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:23.666679 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:23.666848 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:23.669578 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:23.669583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:23.669597 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:23.669623 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:23.671439 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:23.671900 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:23.671906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:23.671912 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:23.672032 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:23.672309 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:23.672313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:23.672315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:23.672711 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:23.672925 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:23.673272 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 10:35:23.673563 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":127,"threads_started":1}
I20260808 10:35:23.674535 10166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dbda3315fe4997b3f36395a6f7bdfb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42547 } }
I20260808 10:35:23.674767 10166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:35:23.674849 10166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:35:23.674952 10166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:23.675944 10166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dbda3315fe4997b3f36395a6f7bdfb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42547 } }
I20260808 10:35:23.676290 10166 leader_election.cc:302] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48dbda3315fe4997b3f36395a6f7bdfb; no voters:
I20260808 10:35:23.676663 10166 leader_election.cc:288] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:35:23.676816 10172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:35:23.677081 10172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [term 1 LEADER]: Becoming Leader. State: Replica: 48dbda3315fe4997b3f36395a6f7bdfb, State: Running, Role: LEADER
I20260808 10:35:23.677408 10172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dbda3315fe4997b3f36395a6f7bdfb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42547 } }
I20260808 10:35:23.677912 10166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:35:23.679088 10174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48dbda3315fe4997b3f36395a6f7bdfb. Latest consensus state: current_term: 1 leader_uuid: "48dbda3315fe4997b3f36395a6f7bdfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dbda3315fe4997b3f36395a6f7bdfb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42547 } } }
I20260808 10:35:23.679235 10174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [sys.catalog]: This master's current role is: LEADER
I20260808 10:35:23.679646 10173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48dbda3315fe4997b3f36395a6f7bdfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dbda3315fe4997b3f36395a6f7bdfb" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42547 } } }
I20260808 10:35:23.680316 10173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb [sys.catalog]: This master's current role is: LEADER
W20260808 10:35:23.682907 10190 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:35:23.682999 10190 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:35:23.683080 10181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:35:23.683857 10181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:35:23.689636 10181 catalog_manager.cc:1347] Generated new cluster ID: 99845500059a40c3a8326160bd7b9063
I20260808 10:35:23.689750 10181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:35:23.703972 10181 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:35:23.705266 10181 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:35:23.722257 10181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb: Generated new TSK 0
I20260808 10:35:23.723145 10181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:35:23.757468 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42547
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:35:23.872834 10195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:23.873126 10195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:23.873261 10195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:23.877440 10195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:23.877504 10195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:23.877595 10195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:35:23.883313 10195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:23.884497 10195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:23.885633 10195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:23.895426 10201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:23.895519 10195 server_base.cc:1034] running on GCE node
W20260808 10:35:23.895426 10202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:23.895421 10205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:23.896114 10195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:23.896708 10195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:23.897881 10195 hybrid_clock.cc:648] HybridClock initialized: now 1786185323897857 us; error 34 us; skew 500 ppm
Aug 08 10:35:23 dist-test-slave-9gcw krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185323, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:23.900764 10195 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:35:23.901746 10195 webserver.cc:460] Webserver started at http://127.30.146.129:42887/ using document root <none> and password file <none>
I20260808 10:35:23.902315 10195 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:23.902379 10195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:23.902632 10195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:23.904670 10195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "60f59c28db2341d78ccb0f09569f9841"
format_stamp: "Formatted at 2026-08-08 10:35:23 on dist-test-slave-9gcw"
server_key: "8f2f191e9b517192b59c47961a12eef2"
server_key_iv: "c38a3c5675bd989490339f51b73958e3"
server_key_version: "encryptionkey@0"
I20260808 10:35:23.905303 10195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "60f59c28db2341d78ccb0f09569f9841"
format_stamp: "Formatted at 2026-08-08 10:35:23 on dist-test-slave-9gcw"
server_key: "8f2f191e9b517192b59c47961a12eef2"
server_key_iv: "c38a3c5675bd989490339f51b73958e3"
server_key_version: "encryptionkey@0"
I20260808 10:35:23.909001 10195 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:35:23.911432 10213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:23.912624 10195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:35:23.912767 10195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "60f59c28db2341d78ccb0f09569f9841"
format_stamp: "Formatted at 2026-08-08 10:35:23 on dist-test-slave-9gcw"
server_key: "8f2f191e9b517192b59c47961a12eef2"
server_key_iv: "c38a3c5675bd989490339f51b73958e3"
server_key_version: "encryptionkey@0"
I20260808 10:35:23.912883 10195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:23.924919 10195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:23.927793 10195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:23.928006 10195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:23.928666 10195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:23.929528 10195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:23.929600 10195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:23.929679 10195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:23.929723 10195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:23.938918 10195 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:44919
I20260808 10:35:23.938961 10338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:44919 every 8 connection(s)
I20260808 10:35:23.939838 10195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:35:23.945322 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 10195
I20260808 10:35:23.945456 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:35:23.945784 31306 external_mini_cluster.cc:1428] Setting key a5053334b17b5bb89fb66dbc3038c4d8
Aug 08 10:35:23 dist-test-slave-9gcw krb5kdc[10078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185323, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:35:23.954752 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:23.941687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:43043 (local address 127.30.146.190:42547)
0808 10:35:23.941855 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:23.941860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:23.942695 (+ 835us) server_negotiation.cc:407] Connection header received
0808 10:35:23.944023 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:23.944026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:23.944070 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:23.944145 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:23.945485 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:23.946091 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:23.947466 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:23.947711 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:23.947824 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:23.950413 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:23.950422 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:23.950424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:23.950449 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:23.952215 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:23.952786 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:23.952792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:23.952796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:23.952903 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:23.953391 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:23.953396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:23.953398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:23.953684 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:23.953806 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:23.954337 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 10:35:23.954492 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:35:23.955463 10341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:23.942081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:42547 (local address 127.30.146.129:43043)
0808 10:35:23.942540 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:23.942582 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:23.943841 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:23.944329 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:23.944336 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:23.944681 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:23.945306 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:23.945328 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:23.946229 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:23.946232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:23.947341 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:23.947350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:23.947866 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:23.947870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:23.948038 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:23.948653 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:23.948668 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:23.950223 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:23.952344 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:23.952349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:23.952361 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:23.952641 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:23.953040 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:23.953043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:23.953045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:23.953282 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:23.953820 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:23.953825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:23.954066 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 10:35:23.954222 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 10:35:23.954509 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
I20260808 10:35:23.956869 10339 heartbeater.cc:347] Connected to a master server at 127.30.146.190:42547
I20260808 10:35:23.957129 10339 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:35:23.957726 10339 heartbeater.cc:511] Master 127.30.146.190:42547 requested a full tablet report, sending...
I20260808 10:35:23.959334 10127 ts_manager.cc:194] Registered new tserver with Master: 60f59c28db2341d78ccb0f09569f9841 (127.30.146.129:44919)
I20260808 10:35:23.960520 10127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:43043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:35:23.978348 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42547
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:35:24.094622 10344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:24.094868 10344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:24.094911 10344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:24.098922 10344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:24.098984 10344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:24.099041 10344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:35:24.104254 10344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:24.105386 10344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:24.106424 10344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:24.116191 10352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:24.116245 10350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:24.116228 10349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:24.116472 10344 server_base.cc:1034] running on GCE node
I20260808 10:35:24.117060 10344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:24.117722 10344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:24.118920 10344 hybrid_clock.cc:648] HybridClock initialized: now 1786185324118906 us; error 37 us; skew 500 ppm
Aug 08 10:35:24 dist-test-slave-9gcw krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185324, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:24.121997 10344 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:35:24.122913 10344 webserver.cc:460] Webserver started at http://127.30.146.130:37441/ using document root <none> and password file <none>
I20260808 10:35:24.123445 10344 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:24.123503 10344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:24.123682 10344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:24.125905 10344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "3ff35c14ff6d4adb9cb0823e5de3188a"
format_stamp: "Formatted at 2026-08-08 10:35:24 on dist-test-slave-9gcw"
server_key: "77775843df540bd7cd12b06fb0cc6b80"
server_key_iv: "70737604250d16e883bb7597eee714ec"
server_key_version: "encryptionkey@0"
I20260808 10:35:24.126423 10344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "3ff35c14ff6d4adb9cb0823e5de3188a"
format_stamp: "Formatted at 2026-08-08 10:35:24 on dist-test-slave-9gcw"
server_key: "77775843df540bd7cd12b06fb0cc6b80"
server_key_iv: "70737604250d16e883bb7597eee714ec"
server_key_version: "encryptionkey@0"
I20260808 10:35:24.130263 10344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:35:24.132666 10360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:24.133952 10344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:35:24.134089 10344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "3ff35c14ff6d4adb9cb0823e5de3188a"
format_stamp: "Formatted at 2026-08-08 10:35:24 on dist-test-slave-9gcw"
server_key: "77775843df540bd7cd12b06fb0cc6b80"
server_key_iv: "70737604250d16e883bb7597eee714ec"
server_key_version: "encryptionkey@0"
I20260808 10:35:24.134198 10344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:24.148751 10344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:24.151590 10344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:24.151804 10344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:24.152400 10344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:24.153321 10344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:24.153370 10344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:24.153434 10344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:24.153473 10344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:24.162989 10344 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:33895
I20260808 10:35:24.163022 10482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:33895 every 8 connection(s)
I20260808 10:35:24.163961 10344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:35:24.165913 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 10344
I20260808 10:35:24.166049 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:35:24.166360 31306 external_mini_cluster.cc:1428] Setting key 5d5d7269f57e21fde7389a459ae641aa
Aug 08 10:35:24 dist-test-slave-9gcw krb5kdc[10078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185324, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:35:24.178546 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.165948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:43713 (local address 127.30.146.190:42547)
0808 10:35:24.166116 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:24.166122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:24.167019 (+ 897us) server_negotiation.cc:407] Connection header received
0808 10:35:24.168431 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:24.168437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:24.168500 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:24.168591 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:24.170072 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.170616 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.171791 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.172032 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.172147 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.174654 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:24.174657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:24.174659 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:24.174690 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:24.176263 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:24.176802 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:24.176810 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:24.176811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:24.176887 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:24.177273 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:24.177276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:24.177278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:24.177506 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:24.177613 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:24.178238 (+ 625us) server_negotiation.cc:299] Negotiation successful
0808 10:35:24.178368 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
I20260808 10:35:24.179621 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.166334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:42547 (local address 127.30.146.130:43713)
0808 10:35:24.166847 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:24.166901 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:24.168200 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:24.168990 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:24.168998 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:24.169433 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:24.169914 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.169927 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.170756 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.170760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.171649 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.171658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.172188 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.172192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.172367 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.172992 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:24.173008 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:24.174483 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:24.176383 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:24.176391 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:24.176408 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:24.176685 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:24.176981 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:24.176984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:24.176986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:24.177130 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:24.177627 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:24.177635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:24.177985 (+ 350us) client_negotiation.cc:769] Sending connection context
0808 10:35:24.178169 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 10:35:24.178503 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":133,"threads_started":1}
I20260808 10:35:24.180877 10483 heartbeater.cc:347] Connected to a master server at 127.30.146.190:42547
I20260808 10:35:24.181130 10483 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:24.181663 10483 heartbeater.cc:511] Master 127.30.146.190:42547 requested a full tablet report, sending...
I20260808 10:35:24.182806 10127 ts_manager.cc:194] Registered new tserver with Master: 3ff35c14ff6d4adb9cb0823e5de3188a (127.30.146.130:33895)
I20260808 10:35:24.183384 10127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:43713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:35:24.196671 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42547
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:35:24.311717 10488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:35:24.311988 10488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:35:24.312083 10488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:35:24.316418 10488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:35:24.316526 10488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:35:24.316591 10488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:35:24.322086 10488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:35:24.323173 10488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:35:24.324229 10488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:35:24.334659 10493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:35:24.334764 10495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:24.334779 10488 server_base.cc:1034] running on GCE node
W20260808 10:35:24.334659 10497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:35:24.335425 10488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:35:24.336078 10488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:35:24.337242 10488 hybrid_clock.cc:648] HybridClock initialized: now 1786185324337224 us; error 49 us; skew 500 ppm
Aug 08 10:35:24 dist-test-slave-9gcw krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185324, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:35:24.340265 10488 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:35:24.341254 10488 webserver.cc:460] Webserver started at http://127.30.146.131:43547/ using document root <none> and password file <none>
I20260808 10:35:24.341828 10488 fs_manager.cc:359] Metadata directory not provided
I20260808 10:35:24.341923 10488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:35:24.342185 10488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:35:24.344252 10488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "65b6c9f8ece64daab588f77b6c53f631"
format_stamp: "Formatted at 2026-08-08 10:35:24 on dist-test-slave-9gcw"
server_key: "0be7d927db2837e60c35ecba37c4776c"
server_key_iv: "ed2616d04762f1986db9303d22a8c700"
server_key_version: "encryptionkey@0"
I20260808 10:35:24.344853 10488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "65b6c9f8ece64daab588f77b6c53f631"
format_stamp: "Formatted at 2026-08-08 10:35:24 on dist-test-slave-9gcw"
server_key: "0be7d927db2837e60c35ecba37c4776c"
server_key_iv: "ed2616d04762f1986db9303d22a8c700"
server_key_version: "encryptionkey@0"
I20260808 10:35:24.348837 10488 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 10:35:24.351709 10506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:24.353250 10488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 10:35:24.353402 10488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "65b6c9f8ece64daab588f77b6c53f631"
format_stamp: "Formatted at 2026-08-08 10:35:24 on dist-test-slave-9gcw"
server_key: "0be7d927db2837e60c35ecba37c4776c"
server_key_iv: "ed2616d04762f1986db9303d22a8c700"
server_key_version: "encryptionkey@0"
I20260808 10:35:24.353525 10488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:35:24.372206 10488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:35:24.375999 10488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:35:24.376255 10488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:35:24.376927 10488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:35:24.377913 10488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:35:24.377992 10488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:24.378074 10488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:35:24.378137 10488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:35:24.388203 10488 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:36885
I20260808 10:35:24.388223 10629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:36885 every 8 connection(s)
I20260808 10:35:24.389314 10488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:35:24.394011 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 10488
I20260808 10:35:24.394160 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:35:24.394548 31306 external_mini_cluster.cc:1428] Setting key 21cdf30df1021dcc261fc6901dee5d46
Aug 08 10:35:24 dist-test-slave-9gcw krb5kdc[10078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185324, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:35:24.404393 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.391432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:43797 (local address 127.30.146.190:42547)
0808 10:35:24.391623 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:24.391627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:24.392531 (+ 904us) server_negotiation.cc:407] Connection header received
0808 10:35:24.394199 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:24.394204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:24.394280 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:24.394384 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:24.396125 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.396724 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.397953 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.398197 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.398310 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.400856 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:24.400859 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:24.400861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:24.400886 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:24.402172 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:24.402778 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:24.402783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:24.402814 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:24.402891 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:24.403245 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:24.403250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:24.403251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:24.403456 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:24.403575 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:24.404070 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 10:35:24.404208 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 10:35:24.405236 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.391838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:42547 (local address 127.30.146.131:43797)
0808 10:35:24.392362 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:24.392406 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:24.393932 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:24.394809 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:24.394816 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:24.395333 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:35:24.395980 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.395997 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.396861 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.396864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.397821 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.397830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.398352 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.398356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.398517 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.399090 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:35:24.399108 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:35:24.400685 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:35:24.402324 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:24.402330 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:24.402345 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:24.402673 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:24.403001 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:35:24.403006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:35:24.403008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:35:24.403150 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:35:24.403575 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:35:24.403593 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:35:24.403839 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 10:35:24.404027 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 10:35:24.404350 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":122,"threads_started":1}
I20260808 10:35:24.406421 10630 heartbeater.cc:347] Connected to a master server at 127.30.146.190:42547
I20260808 10:35:24.406703 10630 heartbeater.cc:464] Registering TS with master...
I20260808 10:35:24.407269 10630 heartbeater.cc:511] Master 127.30.146.190:42547 requested a full tablet report, sending...
I20260808 10:35:24.408401 10126 ts_manager.cc:194] Registered new tserver with Master: 65b6c9f8ece64daab588f77b6c53f631 (127.30.146.131:36885)
I20260808 10:35:24.409015 10126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:43797
I20260808 10:35:24.420663 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:35:24.430522 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.422817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45380 (local address 127.30.146.190:42547)
0808 10:35:24.422984 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:24.422989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:24.423108 (+ 119us) server_negotiation.cc:407] Connection header received
0808 10:35:24.423197 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:24.423199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:24.423241 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:24.423307 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:35:24.424009 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.424534 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.425585 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.425845 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.425937 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.427353 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:35:24.427356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:35:24.427358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:35:24.427382 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:35:24.428985 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:24.429426 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:24.429429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:24.429430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:24.429508 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:35:24.429739 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:35:24.429741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:35:24.429742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:35:24.429956 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:35:24.430067 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:24.430231 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 10:35:24.430325 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:35:24.433645 10126 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:35:24.436693 10126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:35:24.450320 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.445001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:44919 (local address 127.0.0.1:39042)
0808 10:35:24.445371 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:24.445418 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:24.445538 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:24.446078 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:24.446083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:24.446103 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:35:24.446272 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.446280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.447617 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.447623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.448526 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.448536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.449730 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.449736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.449905 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.449947 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 10:35:24.450046 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 10:35:24.450197 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":21,"thread_start_us":101,"threads_started":1}
I20260808 10:35:24.451051 10642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.445176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39042 (local address 127.30.146.129:44919)
0808 10:35:24.445566 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:24.445585 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:24.445615 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:35:24.445700 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:24.445705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:24.445936 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:24.446090 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:35:24.446540 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.447479 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.448659 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.449557 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.450713 (+ 1156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.450822 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:24.450865 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 10:35:24.450939 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":67,"threads_started":1}
I20260808 10:35:24.451488 10644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.445762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:36885 (local address 127.0.0.1:50312)
0808 10:35:24.446482 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:24.446496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:24.446576 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:24.447282 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:24.447284 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:24.447302 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:35:24.447372 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.447380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.448886 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.448891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.450094 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.450124 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.451154 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.451158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.451240 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.451262 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:35:24.451317 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:35:24.451388 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":94,"thread_start_us":92,"threads_started":1}
I20260808 10:35:24.451851 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.445806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49890 (local address 127.30.146.130:33895)
0808 10:35:24.446568 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:24.446580 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:24.446603 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:35:24.446681 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:24.446688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:24.446874 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:24.447338 (+ 464us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:35:24.447510 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.448670 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.450167 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.451012 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.451153 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.451289 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:24.451601 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 10:35:24.451701 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":112,"threads_started":1}
I20260808 10:35:24.452147 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.445306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:33895 (local address 127.0.0.1:49890)
0808 10:35:24.445848 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:24.445862 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:24.445947 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:24.447028 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:24.447031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:24.447049 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:35:24.447171 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.447182 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.448807 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.448810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.450050 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.450059 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.451338 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.451342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.451441 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.451465 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:35:24.451545 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:35:24.451778 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":34,"thread_start_us":104,"threads_started":1}
I20260808 10:35:24.452150 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.446211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.30.146.131:36885)
0808 10:35:24.446835 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:24.446847 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:24.446869 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:35:24.446946 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:24.446954 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:24.447119 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:24.447514 (+ 395us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:35:24.447663 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.448758 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.450257 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.451020 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.451697 (+ 677us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.451835 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:24.451888 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 10:35:24.451984 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":126,"threads_started":1}
I20260808 10:35:24.454334 10265 tablet_service.cc:1467] Processing CreateTablet for tablet e03a2327ebc04931ae88531de8ec7dc4 (DEFAULT_TABLE table=test-table [id=f59c4833a78f456fbdc4cff252081b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:35:24.454953 10561 tablet_service.cc:1467] Processing CreateTablet for tablet e03a2327ebc04931ae88531de8ec7dc4 (DEFAULT_TABLE table=test-table [id=f59c4833a78f456fbdc4cff252081b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:35:24.454953 10414 tablet_service.cc:1467] Processing CreateTablet for tablet e03a2327ebc04931ae88531de8ec7dc4 (DEFAULT_TABLE table=test-table [id=f59c4833a78f456fbdc4cff252081b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:35:24.455555 10265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e03a2327ebc04931ae88531de8ec7dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:24.456061 10414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e03a2327ebc04931ae88531de8ec7dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:24.456066 10561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e03a2327ebc04931ae88531de8ec7dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:35:24.462460 10647 tablet_bootstrap.cc:492] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841: Bootstrap starting.
I20260808 10:35:24.463971 10649 tablet_bootstrap.cc:492] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: Bootstrap starting.
I20260808 10:35:24.464916 10647 tablet_bootstrap.cc:654] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:24.465516 10648 tablet_bootstrap.cc:492] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: Bootstrap starting.
I20260808 10:35:24.465826 10647 log.cc:824] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841: Log is configured to *not* fsync() on all Append() calls
I20260808 10:35:24.466045 10649 tablet_bootstrap.cc:654] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:24.467394 10649 log.cc:824] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: Log is configured to *not* fsync() on all Append() calls
I20260808 10:35:24.467970 10647 tablet_bootstrap.cc:492] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841: No bootstrap required, opened a new log
I20260808 10:35:24.468176 10647 ts_tablet_manager.cc:1397] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:35:24.468251 10648 tablet_bootstrap.cc:654] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: Neither blocks nor log segments found. Creating new log.
I20260808 10:35:24.469080 10648 log.cc:824] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:35:24.469717 10649 tablet_bootstrap.cc:492] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: No bootstrap required, opened a new log
I20260808 10:35:24.469918 10649 ts_tablet_manager.cc:1397] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:35:24.471261 10648 tablet_bootstrap.cc:492] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: No bootstrap required, opened a new log
I20260808 10:35:24.471446 10648 ts_tablet_manager.cc:1397] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:35:24.472247 10647 raft_consensus.cc:348] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 } } peers { permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 } } peers { permanent_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 33895 } }
I20260808 10:35:24.472498 10647 raft_consensus.cc:374] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:24.472553 10647 raft_consensus.cc:729] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60f59c28db2341d78ccb0f09569f9841, State: Initialized, Role: FOLLOWER
I20260808 10:35:24.473014 10647 consensus_queue.cc:260] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 } } peers { permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 } } peers { permanent_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 33895 } }
I20260808 10:35:24.473793 10339 heartbeater.cc:503] Master 127.30.146.190:42547 was elected leader, sending a full tablet report...
I20260808 10:35:24.474143 10647 ts_tablet_manager.cc:1428] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260808 10:35:24.475131 10649 raft_consensus.cc:348] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 } } peers { permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 } } peers { permanent_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 33895 } }
I20260808 10:35:24.475153 10648 raft_consensus.cc:348] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 } } peers { permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 } } peers { permanent_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 33895 } }
I20260808 10:35:24.475442 10649 raft_consensus.cc:374] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:24.475442 10648 raft_consensus.cc:374] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:35:24.475512 10649 raft_consensus.cc:729] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65b6c9f8ece64daab588f77b6c53f631, State: Initialized, Role: FOLLOWER
I20260808 10:35:24.475513 10648 raft_consensus.cc:729] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff35c14ff6d4adb9cb0823e5de3188a, State: Initialized, Role: FOLLOWER
I20260808 10:35:24.475981 10648 consensus_queue.cc:260] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 } } peers { permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 } } peers { permanent_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 33895 } }
I20260808 10:35:24.475981 10649 consensus_queue.cc:260] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 } } peers { permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 } } peers { permanent_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 33895 } }
I20260808 10:35:24.476666 10630 heartbeater.cc:503] Master 127.30.146.190:42547 was elected leader, sending a full tablet report...
I20260808 10:35:24.477008 10483 heartbeater.cc:503] Master 127.30.146.190:42547 was elected leader, sending a full tablet report...
I20260808 10:35:24.477011 10648 ts_tablet_manager.cc:1428] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:35:24.478447 10649 ts_tablet_manager.cc:1428] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 10:35:24.617697 10656 raft_consensus.cc:482] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:35:24.617961 10656 raft_consensus.cc:504] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 } } peers { permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 } } peers { permanent_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 33895 } }
I20260808 10:35:24.619203 10656 leader_election.cc:288] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60f59c28db2341d78ccb0f09569f9841 (127.30.146.129:44919), 65b6c9f8ece64daab588f77b6c53f631 (127.30.146.131:36885)
I20260808 10:35:24.623428 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.619497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:44919 (local address 127.30.146.130:57151)
0808 10:35:24.619636 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:24.619653 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:24.619765 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:24.620161 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:24.620164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:24.620213 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:35:24.620358 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.620365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.621306 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.621312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.622217 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.622227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.622971 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.622975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.623085 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.623113 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 10:35:24.623211 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:35:24.623288 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 10:35:24.623428 10642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.619643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:57151 (local address 127.30.146.129:44919)
0808 10:35:24.619783 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:24.619787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:24.619829 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:35:24.619880 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:24.619902 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:24.619963 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:24.620043 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:35:24.620484 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.621194 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.622347 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.622845 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.622971 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.623029 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:24.623228 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 10:35:24.623299 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:35:24.623631 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.620078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:43347 (local address 127.30.146.131:36885)
0808 10:35:24.620286 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:24.620290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:24.620302 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:35:24.620349 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:24.620351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:24.620389 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:24.620482 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:35:24.620775 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.621595 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.622666 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.623076 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.623206 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.623259 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:24.623452 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 10:35:24.623522 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260808 10:35:24.623647 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.619806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:36885 (local address 127.30.146.130:43347)
0808 10:35:24.620176 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:24.620189 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:35:24.620279 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:35:24.620500 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:35:24.620503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:35:24.620522 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:35:24.620647 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.620653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.621702 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.621709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.622546 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:35:24.622555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.623201 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.623205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:35:24.623328 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.623347 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:35:24.623434 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:35:24.623504 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":160,"threads_started":1}
I20260808 10:35:24.624485 10287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03a2327ebc04931ae88531de8ec7dc4" candidate_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60f59c28db2341d78ccb0f09569f9841" is_pre_election: true
I20260808 10:35:24.624738 10583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03a2327ebc04931ae88531de8ec7dc4" candidate_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65b6c9f8ece64daab588f77b6c53f631" is_pre_election: true
I20260808 10:35:24.624859 10287 raft_consensus.cc:2454] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff35c14ff6d4adb9cb0823e5de3188a in term 0.
I20260808 10:35:24.625077 10583 raft_consensus.cc:2454] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff35c14ff6d4adb9cb0823e5de3188a in term 0.
I20260808 10:35:24.625458 10361 leader_election.cc:302] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff35c14ff6d4adb9cb0823e5de3188a, 60f59c28db2341d78ccb0f09569f9841; no voters:
I20260808 10:35:24.625804 10656 raft_consensus.cc:2781] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:35:24.625885 10656 raft_consensus.cc:482] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:35:24.625916 10656 raft_consensus.cc:3037] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:24.627075 10656 raft_consensus.cc:504] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 } } peers { permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 } } peers { permanent_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 33895 } }
I20260808 10:35:24.627446 10656 leader_election.cc:288] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [CANDIDATE]: Term 1 election: Requested vote from peers 60f59c28db2341d78ccb0f09569f9841 (127.30.146.129:44919), 65b6c9f8ece64daab588f77b6c53f631 (127.30.146.131:36885)
I20260808 10:35:24.627924 10287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03a2327ebc04931ae88531de8ec7dc4" candidate_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60f59c28db2341d78ccb0f09569f9841"
I20260808 10:35:24.628048 10583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03a2327ebc04931ae88531de8ec7dc4" candidate_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65b6c9f8ece64daab588f77b6c53f631"
I20260808 10:35:24.628067 10287 raft_consensus.cc:3037] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:24.628194 10583 raft_consensus.cc:3037] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:35:24.629565 10583 raft_consensus.cc:2454] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff35c14ff6d4adb9cb0823e5de3188a in term 1.
I20260808 10:35:24.629565 10287 raft_consensus.cc:2454] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff35c14ff6d4adb9cb0823e5de3188a in term 1.
I20260808 10:35:24.629933 10361 leader_election.cc:302] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff35c14ff6d4adb9cb0823e5de3188a, 60f59c28db2341d78ccb0f09569f9841; no voters:
I20260808 10:35:24.630205 10656 raft_consensus.cc:2781] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:35:24.630471 10656 raft_consensus.cc:686] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 1 LEADER]: Becoming Leader. State: Replica: 3ff35c14ff6d4adb9cb0823e5de3188a, State: Running, Role: LEADER
I20260808 10:35:24.630846 10656 consensus_queue.cc:237] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 } } peers { permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 } } peers { permanent_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 33895 } }
I20260808 10:35:24.634266 10125 catalog_manager.cc:5495] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ff35c14ff6d4adb9cb0823e5de3188a (127.30.146.130). New cstate: current_term: 1 leader_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ff35c14ff6d4adb9cb0823e5de3188a" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 33895 } health_report { overall_health: HEALTHY } } }
W20260808 10:35:24.640334 10631 tablet.cc:2369] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:35:24.665216 10484 tablet.cc:2369] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:35:24.689917 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.686009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49892 (local address 127.30.146.130:33895)
0808 10:35:24.686159 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:35:24.686163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:35:24.686175 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:35:24.686253 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:35:24.686255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:35:24.686311 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:35:24.686401 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:35:24.686670 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.687188 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.688046 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:35:24.688246 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:35:24.688321 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:35:24.688746 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:35:24.689407 (+ 661us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:35:24.689545 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 10:35:24.689658 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 10:35:24.689715 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 10:35:24.691176 10340 tablet.cc:2369] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:35:24.695873 10287 raft_consensus.cc:1261] T e03a2327ebc04931ae88531de8ec7dc4 P 60f59c28db2341d78ccb0f09569f9841 [term 1 FOLLOWER]: Refusing update from remote peer 3ff35c14ff6d4adb9cb0823e5de3188a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:35:24.695873 10583 raft_consensus.cc:1261] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631 [term 1 FOLLOWER]: Refusing update from remote peer 3ff35c14ff6d4adb9cb0823e5de3188a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:35:24.696617 10656 consensus_queue.cc:1035] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [LEADER]: Connected to new peer: Peer: permanent_uuid: "60f59c28db2341d78ccb0f09569f9841" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:35:24.696806 10658 consensus_queue.cc:1035] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [LEADER]: Connected to new peer: Peer: permanent_uuid: "65b6c9f8ece64daab588f77b6c53f631" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:35:24.703485 10661 mvcc.cc:205] Tried to move back new op lower bound from 7316215089946992640 to 7316215089690923008. Current Snapshot: MvccSnapshot[applied={T|T < 7316215089946992640}]
I20260808 10:35:24.712908 10125 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:35:24.713066 10125 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:35:24.715576 10125 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48dbda3315fe4997b3f36395a6f7bdfb: Sending DeleteTablet for 3 replicas of tablet e03a2327ebc04931ae88531de8ec7dc4
I20260808 10:35:24.716491 10414 tablet_service.cc:1514] Processing DeleteTablet for tablet e03a2327ebc04931ae88531de8ec7dc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:24 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:49890
I20260808 10:35:24.716516 10561 tablet_service.cc:1514] Processing DeleteTablet for tablet e03a2327ebc04931ae88531de8ec7dc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:24 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:50312
I20260808 10:35:24.717098 10671 tablet_replica.cc:330] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: stopping tablet replica
I20260808 10:35:24.717183 10265 tablet_service.cc:1514] Processing DeleteTablet for tablet e03a2327ebc04931ae88531de8ec7dc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:35:24 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:39042
I20260808 10:35:24.717538 10671 raft_consensus.cc:2229] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:35:24.717904 10671 raft_consensus.cc:2258] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:24.717913 10670 tablet_replica.cc:330] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: stopping tablet replica
I20260808 10:35:24.718281 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 10195
I20260808 10:35:24.718298 10670 raft_consensus.cc:2229] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:35:24.718726 10670 raft_consensus.cc:2258] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:35:24.720566 10671 ts_tablet_manager.cc:1905] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:35:24.720804 10670 ts_tablet_manager.cc:1905] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:35:24.723711 10670 ts_tablet_manager.cc:1918] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:35:24.723918 10670 log.cc:1196] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/e03a2327ebc04931ae88531de8ec7dc4
I20260808 10:35:24.724349 10670 ts_tablet_manager.cc:1939] T e03a2327ebc04931ae88531de8ec7dc4 P 65b6c9f8ece64daab588f77b6c53f631: Deleting consensus metadata
I20260808 10:35:24.724403 10671 ts_tablet_manager.cc:1918] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:35:24.724527 10671 log.cc:1196] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/e03a2327ebc04931ae88531de8ec7dc4
I20260808 10:35:24.724908 10671 ts_tablet_manager.cc:1939] T e03a2327ebc04931ae88531de8ec7dc4 P 3ff35c14ff6d4adb9cb0823e5de3188a: Deleting consensus metadata
I20260808 10:35:24.725982 10108 catalog_manager.cc:4841] TS 65b6c9f8ece64daab588f77b6c53f631 (127.30.146.131:36885): tablet e03a2327ebc04931ae88531de8ec7dc4 (table test-table [id=f59c4833a78f456fbdc4cff252081b76]) successfully deleted
I20260808 10:35:24.726082 10111 catalog_manager.cc:4841] TS 3ff35c14ff6d4adb9cb0823e5de3188a (127.30.146.130:33895): tablet e03a2327ebc04931ae88531de8ec7dc4 (table test-table [id=f59c4833a78f456fbdc4cff252081b76]) successfully deleted
W20260808 10:35:24.730515 10108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:44919 (error 108)
I20260808 10:35:24.731696 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:35:24.731375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:44919 (local address 127.0.0.1:39056)
0808 10:35:24.731507 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:35:24.731590 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:44919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:35:24.731801 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 10344
W20260808 10:35:24.731881 10108 catalog_manager.cc:4584] TS 60f59c28db2341d78ccb0f09569f9841 (127.30.146.129:44919): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e03a2327ebc04931ae88531de8ec7dc4: Network error: Client connection negotiation failed: client connection to 127.30.146.129:44919: connect: Connection refused (error 111)
I20260808 10:35:24.738760 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 10488
I20260808 10:35:24.745780 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 10092
2026-08-08T10:35:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2533 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:35:24 dist-test-slave-9gcw krb5kdc[10678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:35:24 dist-test-slave-9gcw krb5kdc[10678](info): set up 2 sockets
Aug 08 10:35:24 dist-test-slave-9gcw krb5kdc[10678](info): commencing operation
krb5kdc: starting...
W20260808 10:35:25.860728 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:35:25 dist-test-slave-9gcw krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:35:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:35:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/rangeradmin_127.30.146.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/rangeradmin_127.30.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/rangerlookup_127.30.146.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/rangerlookup_127.30.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/HTTP_127.30.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/HTTP_127.30.146.148@KRBTEST.COM.keytab.
I20260808 10:35:26.063946 31306 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwxv9J_/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/postgres -l logfile start
2026-08-08 10:35:28.803 UTC [10711] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 10:35:28.803 UTC [10711] LOG: listening on IPv4 address "127.30.146.148", port 54141
2026-08-08 10:35:28.807 UTC [10711] LOG: listening on Unix socket "/tmp/.s.PGSQL.54141"
2026-08-08 10:35:28.829 UTC [10714] LOG: database system was shut down at 2026-08-08 10:35:27 UTC
2026-08-08 10:35:28.834 UTC [10711] LOG: database system is ready to accept connections
I20260808 10:35:29.857491 31306 mini_postgres.cc:96] Postgres bound to 54141
2026-08-08 10:35:29.865 UTC [10722] FATAL: database "slave" does not exist
127.30.146.148:54141 - accepting connections
I20260808 10:35:29.866587 31306 mini_ranger.cc:161] Starting Ranger...
I20260808 10:35:29.885874 31306 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 10:35:30.449466 31306 mini_ranger.cc:87] Created ranger Postgres database
I20260808 10:35:30.449575 31306 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-admin
2026-08-08 10:35:30,773 [I] DB FLAVOR :POSTGRES
2026-08-08 10:35:30,773 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:35:30,774 [I] Checking connection..
2026-08-08 10:35:30,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 10:35:31,050 [I] Checking connection passed.
2026-08-08 10:35:31,050 [I] --------- Verifying version history table ---------
2026-08-08 10:35:31,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:35:31,327 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:35:31,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:35:31,608 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:35:31,608 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 10:35:31,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 10:35:31.866 UTC [10791] WARNING: there is no transaction in progress
2026-08-08 10:35:31,874 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 10:35:31,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:35:32,150 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 10:35:32,150 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:35:32,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 10:35:32,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 10:35:32,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9gcw.c.gcp-upstream.internal','N') ;"
2026-08-08 10:35:32,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:35:33,234 [I] Table x_portal_user does not exist in database ranger
2026-08-08 10:35:33,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:35:33,501 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 10:35:33,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:35:33,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:35:34,039 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 10:35:34,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 10:35:35.073 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.090 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.108 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.124 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.142 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.165 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.237 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.244 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.259 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.271 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.283 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.298 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.309 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.317 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.324 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:35.332 UTC [10916] WARNING: there is no transaction in progress
2026-08-08 10:35:36,047 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 10:35:36,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 10:35:36,296 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 10:35:36,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:35:36,557 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 10:35:36,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:35:36,808 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 10:35:36,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:35:37,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:35:37,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9gcw.c.gcp-upstream.internal';"
2026-08-08 10:35:37,594 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 10:35:37,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 10:35:37,854 [I] DB_PATCHES have already been applied
I20260808 10:35:37.860185 31306 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwxv9J_/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 10:35:37.860299 31306 mini_ranger.cc:193] Using host: 127.30.146.148
I20260808 10:35:37.865381 31306 mini_ranger.cc:227] Ranger admin URL: http://127.30.146.148:57757
Aug 08, 2026 10:35:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:35:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:35:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:35:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:35:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:35:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:35:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 10:35:39.132753 31306 mini_ranger.cc:160] Time spent starting Ranger: real 9.266s user 0.005s sys 0.008s
Aug 08, 2026 10:36:02 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:36:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 10:36:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:36:02 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:36:03 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 10:36:04.109567 31306 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/rangerkms_127.30.146.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/rangerkms_127.30.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.146.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.146.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.146.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/HTTP_127.30.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.146.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/HTTP_127.30.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 10:36:04 dist-test-slave-9gcw krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185364, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 10:36:04.210772 31306 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 10:36:04.223062 31306 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 10:36:05.250748 31306 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 10:36:05.250869 31306 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-kms
I20260808 10:36:05.250983 31306 mini_ranger_kms.cc:227] Using postgres at 127.30.146.148:54141
2026-08-08 10:36:05,971 [I] DB FLAVOR :POSTGRES
2026-08-08 10:36:05,972 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:36:05,972 [I] Checking connection
2026-08-08 10:36:05,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 10:36:06,355 [I] connection success
2026-08-08 10:36:06,355 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 10:36:06,355 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 10:36:06,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 10:36:06,606 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 10:36:06,606 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:36:06,606 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 10:36:06,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54141/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 10:36:06,895 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 10:36:07.426458 31306 mini_ranger_kms.cc:310] Created kms service
I20260808 10:36:07.627671 31306 mini_ranger_kms.cc:326] Created kudu user
I20260808 10:36:07.676909 31306 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 10:36:07 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185364, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:36:08.118510 31306 mini_ranger_kms.cc:384] Added ranger policy
I20260808 10:36:08.118677 31306 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 10:36:08.118698 31306 mini_ranger_kms.cc:240] Using host: 127.30.146.148
I20260808 10:36:08.124024 31306 mini_ranger_kms.cc:276] Ranger KMS PID: 11116
I20260808 10:36:08.124239 31306 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.146.148:39081
Aug 08, 2026 10:36:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:36:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:36:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:36:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:36:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:36:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 10:36:10.605935 31306 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.395s user 0.032s sys 0.660s
I20260808 10:36:10.606156 31306 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 10:36:10.606305 31306 mini_ranger_kms.cc:401] 127.30.146.148:39081/kms/v1/keys
Aug 08 10:36:10 dist-test-slave-9gcw krb5kdc[10678](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185370, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.146.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 10:36:11 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185370, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.146.148@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
Aug 08 10:36:11 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185370, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.146.148@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
Aug 08 10:36:14 dist-test-slave-9gcw krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:36:14.289031 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:42805
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:42805
--ranger_config_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:39081/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:36:14.441283 11169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:36:14.441576 11169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:36:14.441627 11169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:36:14.445830 11169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:36:14.445894 11169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:36:14.445914 11169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:36:14.445930 11169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:36:14.445946 11169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:36:14.451915 11169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.146.148:39081/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:42805
--ranger_config_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:42805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:36:14.453076 11169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:36:14.454216 11169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:36:14.465246 11178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:36:14.465327 11181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:36:14.465363 11177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:36:14.465708 11169 server_base.cc:1034] running on GCE node
I20260808 10:36:14.466388 11169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:36:14.467816 11169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:36:14.469022 11169 hybrid_clock.cc:648] HybridClock initialized: now 1786185374468989 us; error 43 us; skew 500 ppm
Aug 08 10:36:14 dist-test-slave-9gcw krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185374, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:36:14.476377 11169 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:36:14.477447 11169 webserver.cc:460] Webserver started at http://127.30.146.190:45049/ using document root <none> and password file <none>
I20260808 10:36:14.478008 11169 fs_manager.cc:359] Metadata directory not provided
I20260808 10:36:14.478063 11169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:36:14.478300 11169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:36:14 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185374, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:36:14.541450 11169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "26abc163550e4c599c085b3f91cd4f92"
format_stamp: "Formatted at 2026-08-08 10:36:14 on dist-test-slave-9gcw"
server_key: "dfd48f65f3f74949bfc183f579e1824e"
server_key_iv: "1752f3695c6dfd261984ca283c5be587"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:14.542577 11169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "26abc163550e4c599c085b3f91cd4f92"
format_stamp: "Formatted at 2026-08-08 10:36:14 on dist-test-slave-9gcw"
server_key: "dfd48f65f3f74949bfc183f579e1824e"
server_key_iv: "1752f3695c6dfd261984ca283c5be587"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:14.546838 11169 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:36:14.565760 11189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:36:14.567324 11169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:36:14.567446 11169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "26abc163550e4c599c085b3f91cd4f92"
format_stamp: "Formatted at 2026-08-08 10:36:14 on dist-test-slave-9gcw"
server_key: "dfd48f65f3f74949bfc183f579e1824e"
server_key_iv: "1752f3695c6dfd261984ca283c5be587"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:14.567620 11169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:36:14.586956 11169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:36:14.590464 11169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:36:14.590761 11169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:36:14.603071 11169 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:42805
I20260808 10:36:14.603091 11245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:42805 every 8 connection(s)
I20260808 10:36:14.604204 11169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:36:14.607338 11246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:36:14.612911 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 11169
I20260808 10:36:14.613029 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:36:14.613392 31306 external_mini_cluster.cc:1428] Setting key f5fea54fd9dd636395eba9df53cba864
I20260808 10:36:14.613716 11246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92: Bootstrap starting.
I20260808 10:36:14.616791 11246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92: Neither blocks nor log segments found. Creating new log.
I20260808 10:36:14.618495 11246 log.cc:824] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92: Log is configured to *not* fsync() on all Append() calls
I20260808 10:36:14.620877 11246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92: No bootstrap required, opened a new log
Aug 08 10:36:14 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:36:14.626179 11246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26abc163550e4c599c085b3f91cd4f92" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42805 } }
I20260808 10:36:14.626456 11246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:36:14.626538 11246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26abc163550e4c599c085b3f91cd4f92, State: Initialized, Role: FOLLOWER
I20260808 10:36:14.626998 11246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26abc163550e4c599c085b3f91cd4f92" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42805 } }
I20260808 10:36:14.627153 11246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:36:14.627234 11246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:36:14.627367 11246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:36:14.628391 11246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26abc163550e4c599c085b3f91cd4f92" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42805 } }
I20260808 10:36:14.628757 11246 leader_election.cc:302] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26abc163550e4c599c085b3f91cd4f92; no voters:
I20260808 10:36:14.629220 11246 leader_election.cc:288] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:36:14.629284 11251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:36:14.629504 11251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [term 1 LEADER]: Becoming Leader. State: Replica: 26abc163550e4c599c085b3f91cd4f92, State: Running, Role: LEADER
I20260808 10:36:14.629885 11251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26abc163550e4c599c085b3f91cd4f92" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42805 } }
I20260808 10:36:14.630587 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:14.614817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56480 (local address 127.30.146.190:42805)
0808 10:36:14.615473 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:14.615518 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:14.615557 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:36:14.616517 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:14.616536 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:14.616935 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:14.617326 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:36:14.618511 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:14.619634 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:14.620906 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:14.621534 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:14.621668 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:14.624865 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:36:14.624872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:36:14.624887 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:36:14.624918 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:36:14.627017 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:36:14.627580 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:36:14.627586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:36:14.627592 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:36:14.627721 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:36:14.628023 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:36:14.628027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:36:14.628029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:36:14.628454 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:36:14.628697 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:14.629083 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 10:36:14.629663 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":271,"threads_started":1}
I20260808 10:36:14.630618 11246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:36:14.631102 11253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26abc163550e4c599c085b3f91cd4f92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26abc163550e4c599c085b3f91cd4f92" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42805 } } }
I20260808 10:36:14.631373 11253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [sys.catalog]: This master's current role is: LEADER
I20260808 10:36:14.631880 11252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26abc163550e4c599c085b3f91cd4f92. Latest consensus state: current_term: 1 leader_uuid: "26abc163550e4c599c085b3f91cd4f92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26abc163550e4c599c085b3f91cd4f92" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42805 } } }
I20260808 10:36:14.631981 11252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92 [sys.catalog]: This master's current role is: LEADER
I20260808 10:36:14.632707 11256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:36:14.636178 11256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:36:14.636713 11246 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:36:14.648419 11256 catalog_manager.cc:1347] Generated new cluster ID: fc0b8b112f7845499a327946ff0ea4bb
I20260808 10:36:14.648507 11256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:36:14.660180 11256 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:36:14.661350 11256 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:36:14.674726 11256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92: Generated new TSK 0
I20260808 10:36:14.675544 11256 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:36:15.664896 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:39081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42805
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:36:15.846462 11296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:36:15.846747 11296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:36:15.846792 11296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:36:15.851164 11296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:36:15.851240 11296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:36:15.851301 11296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:36:15.859227 11296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.146.148:39081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:36:15.860453 11296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:36:15.861616 11296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:36:15.875245 11303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:36:15.875725 11305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:36:15.875952 11302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:36:15.877420 11296 server_base.cc:1034] running on GCE node
I20260808 10:36:15.877990 11296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:36:15.878907 11296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:36:15.880232 11296 hybrid_clock.cc:648] HybridClock initialized: now 1786185375880207 us; error 39 us; skew 500 ppm
Aug 08 10:36:15 dist-test-slave-9gcw krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:36:15.884086 11296 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:36:15.885129 11296 webserver.cc:460] Webserver started at http://127.30.146.129:43057/ using document root <none> and password file <none>
I20260808 10:36:15.885841 11296 fs_manager.cc:359] Metadata directory not provided
I20260808 10:36:15.885936 11296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:36:15.886188 11296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:36:15 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:36:15.904014 11296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3"
format_stamp: "Formatted at 2026-08-08 10:36:15 on dist-test-slave-9gcw"
server_key: "9287bc60164a73a0f246f30c86f5fc05"
server_key_iv: "718b803b240c01e0474904ad2504e5d3"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:15.904850 11296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3"
format_stamp: "Formatted at 2026-08-08 10:36:15 on dist-test-slave-9gcw"
server_key: "9287bc60164a73a0f246f30c86f5fc05"
server_key_iv: "718b803b240c01e0474904ad2504e5d3"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:15.911161 11296 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260808 10:36:15.928447 11314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:36:15.929925 11296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:36:15.930070 11296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3"
format_stamp: "Formatted at 2026-08-08 10:36:15 on dist-test-slave-9gcw"
server_key: "9287bc60164a73a0f246f30c86f5fc05"
server_key_iv: "718b803b240c01e0474904ad2504e5d3"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:15.930228 11296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:36:15.944479 11296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:36:15.947943 11296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:36:15.948166 11296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:36:15.948822 11296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:36:15.949905 11296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:36:15.949978 11296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:36:15.950054 11296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:36:15.950114 11296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:36:15.970518 11296 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:33097
I20260808 10:36:15.972025 11296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:36:15.973888 11439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:33097 every 8 connection(s)
I20260808 10:36:15.976581 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 11296
I20260808 10:36:15.976744 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:36:15.977115 31306 external_mini_cluster.cc:1428] Setting key b8ad964a3c60598ad86cd926acdfd62f
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
Aug 08 10:36:15 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185375, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:36:16.008852 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:15.976092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:51827 (local address 127.30.146.190:42805)
0808 10:36:15.977889 (+ 1797us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:15.977894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:15.977909 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:36:15.977952 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:15.977955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:15.978000 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:15.985260 (+ 7260us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:36:15.986392 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:15.987308 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:15.990575 (+ 3267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:15.990968 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:15.991073 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.003304 (+ 12231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:36:16.003309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:36:16.003312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:36:16.003347 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:36:16.005634 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:36:16.006649 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:36:16.006653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:36:16.006655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:36:16.006751 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:36:16.007272 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:36:16.007277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:36:16.007279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:36:16.007540 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:36:16.007640 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:16.008509 (+ 869us) server_negotiation.cc:299] Negotiation successful
0808 10:36:16.008668 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1715,"thread_start_us":137,"threads_started":1}
I20260808 10:36:16.011058 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:15.974831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:42805 (local address 127.30.146.129:51827)
0808 10:36:15.975602 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0808 10:36:15.975659 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:36:15.977431 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:36:15.978308 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:36:15.978318 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:36:15.985320 (+ 7002us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:36:15.986177 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:15.986208 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:15.987489 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:15.987493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:15.990421 (+ 2928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:15.990440 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:15.991324 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:15.991328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:15.991580 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:15.996806 (+ 5226us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:36:15.996831 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:36:16.002977 (+ 6146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:36:16.005860 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:36:16.005867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:36:16.005887 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:36:16.006510 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:36:16.006867 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:36:16.006870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:36:16.006872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:36:16.007085 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:36:16.007726 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:36:16.007735 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:36:16.008167 (+ 432us) client_negotiation.cc:769] Sending connection context
0808 10:36:16.009461 (+ 1294us) client_negotiation.cc:240] Negotiation successful
0808 10:36:16.009839 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":235,"threads_started":1}
I20260808 10:36:16.012923 11440 heartbeater.cc:347] Connected to a master server at 127.30.146.190:42805
I20260808 10:36:16.013249 11440 heartbeater.cc:464] Registering TS with master...
I20260808 10:36:16.013890 11440 heartbeater.cc:511] Master 127.30.146.190:42805 requested a full tablet report, sending...
I20260808 10:36:16.015816 11207 ts_manager.cc:194] Registered new tserver with Master: 30d6be840f0e4ac1bfd3c28f5d8f45f3 (127.30.146.129:33097)
I20260808 10:36:16.017620 11207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:51827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:36:16.029503 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:39081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42805
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:36:16 dist-test-slave-9gcw krb5kdc[10678](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185376, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 10:36:16.255865 11448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:36:16.256410 11448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:36:16.256532 11448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:36:16.263502 11448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:36:16.263592 11448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:36:16.263775 11448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:36:16.273022 11448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.146.148:39081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:36:16.274657 11448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:36:16.276052 11448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:36:16.291952 11455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:36:16.292251 11448 server_base.cc:1034] running on GCE node
W20260808 10:36:16.291956 11456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:36:16.292613 11458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:36:16.293123 11448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:36:16.294049 11448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:36:16.295264 11448 hybrid_clock.cc:648] HybridClock initialized: now 1786185376295234 us; error 44 us; skew 500 ppm
Aug 08 10:36:16 dist-test-slave-9gcw krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185376, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:36:16.300232 11448 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:36:16.301378 11448 webserver.cc:460] Webserver started at http://127.30.146.130:42531/ using document root <none> and password file <none>
I20260808 10:36:16.301992 11448 fs_manager.cc:359] Metadata directory not provided
I20260808 10:36:16.302062 11448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:36:16.302312 11448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:36:16 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185376, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:36:16.324069 11448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "7c91206ee2e8491389518ee364b1c104"
format_stamp: "Formatted at 2026-08-08 10:36:16 on dist-test-slave-9gcw"
server_key: "63c098d4bc8f2c71038506ffdd1a9e9b"
server_key_iv: "66d669d8ff439599de558be5c4b5cd1e"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:16.324877 11448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "7c91206ee2e8491389518ee364b1c104"
format_stamp: "Formatted at 2026-08-08 10:36:16 on dist-test-slave-9gcw"
server_key: "63c098d4bc8f2c71038506ffdd1a9e9b"
server_key_iv: "66d669d8ff439599de558be5c4b5cd1e"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:16.329850 11448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 10:36:16.351142 11465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:36:16.352989 11448 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:36:16.353205 11448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "7c91206ee2e8491389518ee364b1c104"
format_stamp: "Formatted at 2026-08-08 10:36:16 on dist-test-slave-9gcw"
server_key: "63c098d4bc8f2c71038506ffdd1a9e9b"
server_key_iv: "66d669d8ff439599de558be5c4b5cd1e"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:16.353390 11448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:36:16.370869 11448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:36:16.375303 11448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:36:16.375627 11448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:36:16.376475 11448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:36:16.377743 11448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:36:16.377864 11448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:36:16.377981 11448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:36:16.378059 11448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:36:16.405946 11448 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:42795
I20260808 10:36:16.406214 11591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:42795 every 8 connection(s)
I20260808 10:36:16.407141 11448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:36:16.412232 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 11448
I20260808 10:36:16.412385 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:36:16.412729 31306 external_mini_cluster.cc:1428] Setting key 49eab2fe96a5065b29af2cd5f730b4b1
Aug 08 10:36:16 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185376, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:36:16.429983 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.409494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:42491 (local address 127.30.146.190:42805)
0808 10:36:16.409633 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:16.409638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:16.410717 (+ 1079us) server_negotiation.cc:407] Connection header received
0808 10:36:16.411932 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:16.411936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:16.412002 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:16.412077 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:36:16.417601 (+ 5524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.418470 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.420393 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.420810 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.420923 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.425138 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:36:16.425143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:36:16.425148 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:36:16.425216 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:36:16.427437 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:36:16.428115 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:36:16.428118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:36:16.428120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:36:16.428218 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:36:16.428653 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:36:16.428657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:36:16.428659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:36:16.428864 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:36:16.428977 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:16.429592 (+ 615us) server_negotiation.cc:299] Negotiation successful
0808 10:36:16.429803 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
I20260808 10:36:16.431136 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.409950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:42805 (local address 127.30.146.130:42491)
0808 10:36:16.410525 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0808 10:36:16.410579 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:36:16.411712 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:36:16.413416 (+ 1704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:36:16.413423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:36:16.413946 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:36:16.417405 (+ 3459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:16.417431 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.418624 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.418630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:16.420233 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:16.420250 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.421191 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.421197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:16.421487 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.422373 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:36:16.422403 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:36:16.424872 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:36:16.427586 (+ 2714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:36:16.427595 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:36:16.427615 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:36:16.427976 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:36:16.428325 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:36:16.428331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:36:16.428336 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:36:16.428542 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:36:16.428982 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:36:16.428991 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:36:16.429347 (+ 356us) client_negotiation.cc:769] Sending connection context
0808 10:36:16.429553 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 10:36:16.429911 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":158,"threads_started":1}
I20260808 10:36:16.434142 11592 heartbeater.cc:347] Connected to a master server at 127.30.146.190:42805
I20260808 10:36:16.434420 11592 heartbeater.cc:464] Registering TS with master...
I20260808 10:36:16.434966 11592 heartbeater.cc:511] Master 127.30.146.190:42805 requested a full tablet report, sending...
I20260808 10:36:16.436232 11207 ts_manager.cc:194] Registered new tserver with Master: 7c91206ee2e8491389518ee364b1c104 (127.30.146.130:42795)
I20260808 10:36:16.436830 11207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:42491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:36:16.452678 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:39081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42805
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:36:16.682683 11598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:36:16.683259 11598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:36:16.683418 11598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:36:16.690675 11598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:36:16.690797 11598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:36:16.690905 11598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:36:16.700541 11598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.146.148:39081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:42805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:36:16.702287 11598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:36:16.703884 11598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:36:16.722362 11608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:36:16.723251 11604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:36:16.723568 11598 server_base.cc:1034] running on GCE node
W20260808 10:36:16.723372 11605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:36:16.724267 11598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:36:16.725111 11598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:36:16.726626 11598 hybrid_clock.cc:648] HybridClock initialized: now 1786185376726601 us; error 46 us; skew 500 ppm
Aug 08 10:36:16 dist-test-slave-9gcw krb5kdc[10678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185376, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:36:16.731725 11598 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:36:16.733038 11598 webserver.cc:460] Webserver started at http://127.30.146.131:39653/ using document root <none> and password file <none>
I20260808 10:36:16.733896 11598 fs_manager.cc:359] Metadata directory not provided
I20260808 10:36:16.733999 11598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:36:16.734344 11598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:36:16 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185376, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:36:16.753039 11598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "ce207f038b3b44edad2bc8ea4265a4d6"
format_stamp: "Formatted at 2026-08-08 10:36:16 on dist-test-slave-9gcw"
server_key: "4089964c69b79e5f720af15ed2353612"
server_key_iv: "a2137ca112060c8d2cece594082ee3be"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:16.754287 11598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "ce207f038b3b44edad2bc8ea4265a4d6"
format_stamp: "Formatted at 2026-08-08 10:36:16 on dist-test-slave-9gcw"
server_key: "4089964c69b79e5f720af15ed2353612"
server_key_iv: "a2137ca112060c8d2cece594082ee3be"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:16.763998 11598 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.000s
I20260808 10:36:16.786377 11615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:36:16.788460 11598 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260808 10:36:16.788625 11598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "ce207f038b3b44edad2bc8ea4265a4d6"
format_stamp: "Formatted at 2026-08-08 10:36:16 on dist-test-slave-9gcw"
server_key: "4089964c69b79e5f720af15ed2353612"
server_key_iv: "a2137ca112060c8d2cece594082ee3be"
server_key_version: "kuduclusterkey@0"
I20260808 10:36:16.788797 11598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:36:16.828038 11598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
Aug 08 10:36:16 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185376, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:36:16.842375 11598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:36:16.842659 11598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:36:16.844059 11598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:36:16.845293 11598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:36:16.847678 11598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 10:36:16.847851 11598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:36:16.847947 11598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:36:16.883728 11598 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:42063
I20260808 10:36:16.885133 11598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:36:16.888572 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 11598
I20260808 10:36:16.888702 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:36:16.889024 31306 external_mini_cluster.cc:1428] Setting key 6aa3bc66439db4755820db74f81f1c38
I20260808 10:36:16.898881 11744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:42063 every 8 connection(s)
Aug 08 10:36:16 dist-test-slave-9gcw krb5kdc[10678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185376, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:36:16.908712 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.887541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:51479 (local address 127.30.146.190:42805)
0808 10:36:16.887679 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:16.887685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:16.889688 (+ 2003us) server_negotiation.cc:407] Connection header received
0808 10:36:16.891614 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:16.891617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:16.891673 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:16.891738 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:36:16.893499 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.894555 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.897276 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.897668 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.897752 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.902796 (+ 5044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:36:16.902800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:36:16.902803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:36:16.902835 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:36:16.905659 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:36:16.906508 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:36:16.906512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:36:16.906514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:36:16.906624 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:36:16.907165 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:36:16.907169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:36:16.907172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:36:16.907446 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:36:16.907541 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:16.908357 (+ 816us) server_negotiation.cc:299] Negotiation successful
0808 10:36:16.908515 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:36:16.911268 11747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.888106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:42805 (local address 127.30.146.131:51479)
0808 10:36:16.889522 (+ 1416us) negotiation.cc:107] Waiting for socket to connect
0808 10:36:16.889568 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:36:16.891374 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:36:16.892067 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:36:16.892073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:36:16.892643 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:36:16.893352 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:16.893370 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.895129 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.895133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:16.897033 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:16.897055 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.897987 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.897991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:16.898326 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.899435 (+ 1109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:36:16.899461 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:36:16.902578 (+ 3117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:36:16.905823 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:36:16.905831 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:36:16.905851 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:36:16.906364 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:36:16.906759 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:36:16.906763 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:36:16.906765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:36:16.906993 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:36:16.907622 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:36:16.907629 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:36:16.908010 (+ 381us) client_negotiation.cc:769] Sending connection context
0808 10:36:16.909292 (+ 1282us) client_negotiation.cc:240] Negotiation successful
0808 10:36:16.909707 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1177,"thread_start_us":171,"threads_started":1}
I20260808 10:36:16.913419 11745 heartbeater.cc:347] Connected to a master server at 127.30.146.190:42805
I20260808 10:36:16.913718 11745 heartbeater.cc:464] Registering TS with master...
I20260808 10:36:16.914397 11745 heartbeater.cc:511] Master 127.30.146.190:42805 requested a full tablet report, sending...
I20260808 10:36:16.915696 11207 ts_manager.cc:194] Registered new tserver with Master: ce207f038b3b44edad2bc8ea4265a4d6 (127.30.146.131:42063)
I20260808 10:36:16.916478 11207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:51479
I20260808 10:36:16.916952 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:36:16.933604 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.919962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56482 (local address 127.30.146.190:42805)
0808 10:36:16.920395 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:16.920400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:16.920525 (+ 125us) server_negotiation.cc:407] Connection header received
0808 10:36:16.920717 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:16.920720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:16.920776 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:16.920869 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:36:16.921816 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.922396 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.925114 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.925827 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.926047 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.928725 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:36:16.928731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:36:16.928736 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:36:16.928779 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:36:16.930888 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:36:16.931512 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:36:16.931517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:36:16.931519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:36:16.931639 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:36:16.932038 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:36:16.932043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:36:16.932045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:36:16.932323 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:36:16.932455 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:16.933278 (+ 823us) server_negotiation.cc:299] Negotiation successful
0808 10:36:16.933422 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346}
I20260808 10:36:16.937757 11207 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:36:16.943723 11207 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:36:16.966173 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.956443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:42063 (local address 127.0.0.1:47188)
0808 10:36:16.960087 (+ 3644us) negotiation.cc:107] Waiting for socket to connect
0808 10:36:16.960103 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:36:16.960248 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:36:16.960821 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:36:16.960827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:36:16.960850 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:36:16.961000 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:16.961006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.962827 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.962830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:16.964256 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:16.964268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.965630 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.965634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:16.965749 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.965798 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 10:36:16.965885 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:36:16.966004 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3551,"mutex_wait_us":1183,"thread_start_us":373,"threads_started":1}
I20260808 10:36:16.967142 11759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.956842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47188 (local address 127.30.146.131:42063)
0808 10:36:16.957493 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:16.957510 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:16.960279 (+ 2769us) server_negotiation.cc:407] Connection header received
0808 10:36:16.960385 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:16.960390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:16.960622 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:16.961196 (+ 574us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:36:16.961338 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.962688 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.964571 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.965489 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.966645 (+ 1156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.966798 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:16.966849 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 10:36:16.966947 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":250,"threads_started":1}
I20260808 10:36:16.970352 11764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.957615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:33097 (local address 127.0.0.1:42840)
0808 10:36:16.959639 (+ 2024us) negotiation.cc:107] Waiting for socket to connect
0808 10:36:16.959655 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:36:16.959781 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:36:16.962579 (+ 2798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:36:16.962582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:36:16.962605 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:36:16.962731 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:16.962741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.965992 (+ 3251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.965996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:16.967495 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:16.967508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.969836 (+ 2328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.969841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:16.969951 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.969984 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:36:16.970090 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 10:36:16.970184 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1948,"thread_start_us":310,"threads_started":1}
I20260808 10:36:16.971596 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.957758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42840 (local address 127.30.146.129:33097)
0808 10:36:16.960554 (+ 2796us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:16.960562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:16.960591 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:36:16.960698 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:16.960705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:16.962367 (+ 1662us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:16.962998 (+ 631us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:36:16.963196 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.964666 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.967924 (+ 3258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.969345 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.970979 (+ 1634us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.971194 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:16.971263 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 10:36:16.971379 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2663,"thread_start_us":163,"threads_started":1}
I20260808 10:36:16.972391 11662 tablet_service.cc:1467] Processing CreateTablet for tablet cf634df344cc4b389b97f6fd6d5f6a82 (DEFAULT_TABLE table=test-table [id=0814dc9be6924331a52c2ec4221bf381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:36:16.974751 11662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf634df344cc4b389b97f6fd6d5f6a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:36:16.975813 11369 tablet_service.cc:1467] Processing CreateTablet for tablet cf634df344cc4b389b97f6fd6d5f6a82 (DEFAULT_TABLE table=test-table [id=0814dc9be6924331a52c2ec4221bf381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:36:16.977895 11369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf634df344cc4b389b97f6fd6d5f6a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:36:16.987432 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.956441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:42795 (local address 127.0.0.1:60846)
0808 10:36:16.959057 (+ 2616us) negotiation.cc:107] Waiting for socket to connect
0808 10:36:16.959141 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0808 10:36:16.959302 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:36:16.966004 (+ 6702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:36:16.966007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:36:16.966047 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:36:16.966285 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:16.966295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.968101 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.968105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:16.973453 (+ 5348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:16.973469 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.986750 (+ 13281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.986755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:16.986962 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.986999 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 10:36:16.987082 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:36:16.987213 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2235,"thread_start_us":223,"threads_started":1}
I20260808 10:36:16.988519 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:16.957009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60846 (local address 127.30.146.130:42795)
0808 10:36:16.965409 (+ 8400us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:16.965419 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:16.965450 (+ 31us) server_negotiation.cc:407] Connection header received
0808 10:36:16.965546 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:16.965554 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:16.965822 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:16.966492 (+ 670us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:36:16.966708 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.967933 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.985332 (+ 17399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:16.986497 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:16.988039 (+ 1542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:16.988196 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:16.988252 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 10:36:16.988350 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8258,"thread_start_us":151,"threads_started":1}
I20260808 10:36:16.991420 11765 tablet_bootstrap.cc:492] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3: Bootstrap starting.
I20260808 10:36:16.992492 11516 tablet_service.cc:1467] Processing CreateTablet for tablet cf634df344cc4b389b97f6fd6d5f6a82 (DEFAULT_TABLE table=test-table [id=0814dc9be6924331a52c2ec4221bf381]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:36:16.994572 11766 tablet_bootstrap.cc:492] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: Bootstrap starting.
I20260808 10:36:16.994781 11516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf634df344cc4b389b97f6fd6d5f6a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:36:16.999609 11766 tablet_bootstrap.cc:654] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: Neither blocks nor log segments found. Creating new log.
I20260808 10:36:17.000677 11765 tablet_bootstrap.cc:654] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3: Neither blocks nor log segments found. Creating new log.
I20260808 10:36:17.002463 11765 log.cc:824] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:36:17.004843 11766 log.cc:824] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:36:17.007870 11765 tablet_bootstrap.cc:492] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3: No bootstrap required, opened a new log
I20260808 10:36:17.008222 11765 ts_tablet_manager.cc:1397] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.004s
I20260808 10:36:17.012084 11766 tablet_bootstrap.cc:492] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: No bootstrap required, opened a new log
I20260808 10:36:17.013816 11766 ts_tablet_manager.cc:1397] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.005s
I20260808 10:36:17.015528 11765 raft_consensus.cc:348] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 } } peers { permanent_uuid: "7c91206ee2e8491389518ee364b1c104" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42795 } } peers { permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 } }
I20260808 10:36:17.016135 11765 raft_consensus.cc:374] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:36:17.016278 11765 raft_consensus.cc:729] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d6be840f0e4ac1bfd3c28f5d8f45f3, State: Initialized, Role: FOLLOWER
I20260808 10:36:17.017079 11765 consensus_queue.cc:260] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 } } peers { permanent_uuid: "7c91206ee2e8491389518ee364b1c104" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42795 } } peers { permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 } }
I20260808 10:36:17.018726 11765 ts_tablet_manager.cc:1428] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260808 10:36:17.019285 11440 heartbeater.cc:503] Master 127.30.146.190:42805 was elected leader, sending a full tablet report...
I20260808 10:36:17.021037 11766 raft_consensus.cc:348] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 } } peers { permanent_uuid: "7c91206ee2e8491389518ee364b1c104" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42795 } } peers { permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 } }
I20260808 10:36:17.021344 11766 raft_consensus.cc:374] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:36:17.021404 11766 raft_consensus.cc:729] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce207f038b3b44edad2bc8ea4265a4d6, State: Initialized, Role: FOLLOWER
I20260808 10:36:17.021842 11766 consensus_queue.cc:260] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 } } peers { permanent_uuid: "7c91206ee2e8491389518ee364b1c104" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42795 } } peers { permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 } }
I20260808 10:36:17.023046 11766 ts_tablet_manager.cc:1428] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260808 10:36:17.023734 11769 tablet_bootstrap.cc:492] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: Bootstrap starting.
I20260808 10:36:17.027827 11769 tablet_bootstrap.cc:654] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: Neither blocks nor log segments found. Creating new log.
I20260808 10:36:17.029235 11769 log.cc:824] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: Log is configured to *not* fsync() on all Append() calls
I20260808 10:36:17.030009 11745 heartbeater.cc:503] Master 127.30.146.190:42805 was elected leader, sending a full tablet report...
I20260808 10:36:17.032970 11769 tablet_bootstrap.cc:492] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: No bootstrap required, opened a new log
I20260808 10:36:17.033275 11769 ts_tablet_manager.cc:1397] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260808 10:36:17.039471 11769 raft_consensus.cc:348] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 } } peers { permanent_uuid: "7c91206ee2e8491389518ee364b1c104" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42795 } } peers { permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 } }
I20260808 10:36:17.039769 11769 raft_consensus.cc:374] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:36:17.039878 11769 raft_consensus.cc:729] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c91206ee2e8491389518ee364b1c104, State: Initialized, Role: FOLLOWER
I20260808 10:36:17.040443 11769 consensus_queue.cc:260] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 } } peers { permanent_uuid: "7c91206ee2e8491389518ee364b1c104" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42795 } } peers { permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 } }
I20260808 10:36:17.041426 11592 heartbeater.cc:503] Master 127.30.146.190:42805 was elected leader, sending a full tablet report...
I20260808 10:36:17.043313 11769 ts_tablet_manager.cc:1428] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
W20260808 10:36:17.138645 11746 tablet.cc:2369] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:36:17.153070 11773 raft_consensus.cc:482] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:36:17.153457 11773 raft_consensus.cc:504] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 } } peers { permanent_uuid: "7c91206ee2e8491389518ee364b1c104" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42795 } } peers { permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 } }
I20260808 10:36:17.155010 11773 leader_election.cc:288] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce207f038b3b44edad2bc8ea4265a4d6 (127.30.146.131:42063), 30d6be840f0e4ac1bfd3c28f5d8f45f3 (127.30.146.129:33097)
W20260808 10:36:17.160022 11593 tablet.cc:2369] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:36:17.162889 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:17.156239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:36099 (local address 127.30.146.129:33097)
0808 10:36:17.156385 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:17.156390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:17.156873 (+ 483us) server_negotiation.cc:407] Connection header received
0808 10:36:17.157199 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:17.157203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:17.157263 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:17.157323 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:36:17.158068 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:17.159021 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:17.161288 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:17.161915 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:17.162017 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:17.162094 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:17.162658 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 10:36:17.162747 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:36:17.163228 11759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:17.155795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:40357 (local address 127.30.146.131:42063)
0808 10:36:17.155922 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:17.155926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:17.156365 (+ 439us) server_negotiation.cc:407] Connection header received
0808 10:36:17.156588 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:17.156591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:17.156645 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:17.156695 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:36:17.157538 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:17.158360 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:17.161769 (+ 3409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:17.162415 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:17.162517 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:17.162566 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:17.163002 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 10:36:17.163089 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:36:17.163844 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:17.155538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:33097 (local address 127.30.146.130:36099)
0808 10:36:17.156713 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0808 10:36:17.156730 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:36:17.156961 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:36:17.157493 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:36:17.157496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:36:17.157516 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:36:17.157860 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:17.157870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:17.159256 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:17.159260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:17.161089 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:17.161101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:17.162301 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:17.162304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:17.162460 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:17.162486 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:36:17.163500 (+ 1014us) client_negotiation.cc:240] Negotiation successful
0808 10:36:17.163597 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1070,"mutex_wait_us":5,"thread_start_us":186,"threads_started":1}
I20260808 10:36:17.163861 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:17.155364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:42063 (local address 127.30.146.130:40357)
0808 10:36:17.156180 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0808 10:36:17.156195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:36:17.156456 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:36:17.156839 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:36:17.156842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:36:17.156859 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:36:17.157391 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:17.157402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:17.158718 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:17.158722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:17.161604 (+ 2882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:36:17.161616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:36:17.162665 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:36:17.162668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:36:17.162849 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:17.162873 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:36:17.163484 (+ 611us) client_negotiation.cc:240] Negotiation successful
0808 10:36:17.163576 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":11,"thread_start_us":139,"threads_started":1}
I20260808 10:36:17.164801 11390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf634df344cc4b389b97f6fd6d5f6a82" candidate_uuid: "7c91206ee2e8491389518ee364b1c104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" is_pre_election: true
I20260808 10:36:17.165076 11672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf634df344cc4b389b97f6fd6d5f6a82" candidate_uuid: "7c91206ee2e8491389518ee364b1c104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" is_pre_election: true
I20260808 10:36:17.165207 11390 raft_consensus.cc:2454] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c91206ee2e8491389518ee364b1c104 in term 0.
I20260808 10:36:17.165396 11672 raft_consensus.cc:2454] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c91206ee2e8491389518ee364b1c104 in term 0.
I20260808 10:36:17.166018 11467 leader_election.cc:302] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c91206ee2e8491389518ee364b1c104, ce207f038b3b44edad2bc8ea4265a4d6; no voters:
I20260808 10:36:17.166378 11773 raft_consensus.cc:2781] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:36:17.166476 11773 raft_consensus.cc:482] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:36:17.166522 11773 raft_consensus.cc:3037] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:36:17.168570 11773 raft_consensus.cc:504] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 } } peers { permanent_uuid: "7c91206ee2e8491389518ee364b1c104" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42795 } } peers { permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 } }
I20260808 10:36:17.168972 11773 leader_election.cc:288] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [CANDIDATE]: Term 1 election: Requested vote from peers ce207f038b3b44edad2bc8ea4265a4d6 (127.30.146.131:42063), 30d6be840f0e4ac1bfd3c28f5d8f45f3 (127.30.146.129:33097)
I20260808 10:36:17.169368 11672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf634df344cc4b389b97f6fd6d5f6a82" candidate_uuid: "7c91206ee2e8491389518ee364b1c104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce207f038b3b44edad2bc8ea4265a4d6"
I20260808 10:36:17.169502 11672 raft_consensus.cc:3037] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:36:17.169651 11390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf634df344cc4b389b97f6fd6d5f6a82" candidate_uuid: "7c91206ee2e8491389518ee364b1c104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3"
I20260808 10:36:17.169792 11390 raft_consensus.cc:3037] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:36:17.171563 11672 raft_consensus.cc:2454] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c91206ee2e8491389518ee364b1c104 in term 1.
I20260808 10:36:17.171952 11467 leader_election.cc:302] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c91206ee2e8491389518ee364b1c104, ce207f038b3b44edad2bc8ea4265a4d6; no voters:
I20260808 10:36:17.171834 11390 raft_consensus.cc:2454] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c91206ee2e8491389518ee364b1c104 in term 1.
I20260808 10:36:17.172227 11773 raft_consensus.cc:2781] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:36:17.172474 11773 raft_consensus.cc:686] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 1 LEADER]: Becoming Leader. State: Replica: 7c91206ee2e8491389518ee364b1c104, State: Running, Role: LEADER
I20260808 10:36:17.172925 11773 consensus_queue.cc:237] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 } } peers { permanent_uuid: "7c91206ee2e8491389518ee364b1c104" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42795 } } peers { permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 } }
I20260808 10:36:17.176949 11207 catalog_manager.cc:5495] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c91206ee2e8491389518ee364b1c104 (127.30.146.130). New cstate: current_term: 1 leader_uuid: "7c91206ee2e8491389518ee364b1c104" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c91206ee2e8491389518ee364b1c104" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 } health_report { overall_health: UNKNOWN } } }
I20260808 10:36:17.211829 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:17.206433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.30.146.130:42795)
0808 10:36:17.206597 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:36:17.206602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:36:17.206619 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:36:17.206711 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:36:17.206714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:36:17.206780 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:36:17.206896 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:36:17.207418 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:17.208211 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:17.209547 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:36:17.209887 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:36:17.209994 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:36:17.210442 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:36:17.211026 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:36:17.211149 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 10:36:17.211606 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 10:36:17.211675 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:36:17.220388 11390 raft_consensus.cc:1261] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3 [term 1 FOLLOWER]: Refusing update from remote peer 7c91206ee2e8491389518ee364b1c104: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:36:17.220608 11672 raft_consensus.cc:1261] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6 [term 1 FOLLOWER]: Refusing update from remote peer 7c91206ee2e8491389518ee364b1c104: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:36:17.221448 11773 consensus_queue.cc:1035] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d6be840f0e4ac1bfd3c28f5d8f45f3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 33097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:36:17.222008 11776 consensus_queue.cc:1035] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce207f038b3b44edad2bc8ea4265a4d6" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:36:17.224833 11441 tablet.cc:2369] T cf634df344cc4b389b97f6fd6d5f6a82 P 30d6be840f0e4ac1bfd3c28f5d8f45f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:36:17.232887 11781 mvcc.cc:205] Tried to move back new op lower bound from 7316215305086140416 to 7316215304903712768. Current Snapshot: MvccSnapshot[applied={T|T < 7316215305086140416}]
I20260808 10:36:17.233399 11780 mvcc.cc:205] Tried to move back new op lower bound from 7316215305086140416 to 7316215304903712768. Current Snapshot: MvccSnapshot[applied={T|T < 7316215305086140416}]
I20260808 10:36:17.242823 11782 mvcc.cc:205] Tried to move back new op lower bound from 7316215305086140416 to 7316215304903712768. Current Snapshot: MvccSnapshot[applied={T|T < 7316215305086140416}]
I20260808 10:36:17.252737 11206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:36:17.252978 11206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:36:17.256592 11206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26abc163550e4c599c085b3f91cd4f92: Sending DeleteTablet for 3 replicas of tablet cf634df344cc4b389b97f6fd6d5f6a82
I20260808 10:36:17.257766 11662 tablet_service.cc:1514] Processing DeleteTablet for tablet cf634df344cc4b389b97f6fd6d5f6a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:36:17 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:47188
I20260808 10:36:17.258536 11516 tablet_service.cc:1514] Processing DeleteTablet for tablet cf634df344cc4b389b97f6fd6d5f6a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:36:17 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:60846
I20260808 10:36:17.259968 11369 tablet_service.cc:1514] Processing DeleteTablet for tablet cf634df344cc4b389b97f6fd6d5f6a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:36:17 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:42840
I20260808 10:36:17.261921 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 11296
I20260808 10:36:17.263281 11790 tablet_replica.cc:330] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: stopping tablet replica
I20260808 10:36:17.265539 11790 raft_consensus.cc:2229] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:36:17.266083 11790 raft_consensus.cc:2258] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:36:17.266532 11789 tablet_replica.cc:330] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: stopping tablet replica
I20260808 10:36:17.267143 11789 raft_consensus.cc:2229] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:36:17.268676 11790 ts_tablet_manager.cc:1905] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:36:17.269105 11789 raft_consensus.cc:2258] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:36:17.277712 11790 ts_tablet_manager.cc:1918] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:36:17.279687 11789 ts_tablet_manager.cc:1905] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:36:17.280272 11192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:33097 (error 108)
I20260808 10:36:17.283320 11789 ts_tablet_manager.cc:1918] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:36:17.284439 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:17.284081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:33097 (local address 127.0.0.1:42852)
0808 10:36:17.284225 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:36:17.284325 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:33097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 10:36:17.284700 11790 log.cc:1196] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/cf634df344cc4b389b97f6fd6d5f6a82
W20260808 10:36:17.284680 11192 catalog_manager.cc:4584] TS 30d6be840f0e4ac1bfd3c28f5d8f45f3 (127.30.146.129:33097): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf634df344cc4b389b97f6fd6d5f6a82: Network error: Client connection negotiation failed: client connection to 127.30.146.129:33097: connect: Connection refused (error 111)
I20260808 10:36:17.285104 11790 ts_tablet_manager.cc:1939] T cf634df344cc4b389b97f6fd6d5f6a82 P 7c91206ee2e8491389518ee364b1c104: Deleting consensus metadata
I20260808 10:36:17.285408 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 11448
I20260808 10:36:17.286257 11789 log.cc:1196] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/cf634df344cc4b389b97f6fd6d5f6a82
I20260808 10:36:17.286576 11789 ts_tablet_manager.cc:1939] T cf634df344cc4b389b97f6fd6d5f6a82 P ce207f038b3b44edad2bc8ea4265a4d6: Deleting consensus metadata
I20260808 10:36:17.288098 11190 catalog_manager.cc:4841] TS ce207f038b3b44edad2bc8ea4265a4d6 (127.30.146.131:42063): tablet cf634df344cc4b389b97f6fd6d5f6a82 (table test-table [id=0814dc9be6924331a52c2ec4221bf381]) successfully deleted
I20260808 10:36:17.296113 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 11598
I20260808 10:36:17.297631 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:36:17.296307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:42795 (local address 127.0.0.1:60874)
0808 10:36:17.297412 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0808 10:36:17.297496 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.130:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":974}
I20260808 10:36:17.305253 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 11169
I20260808 10:36:17.315611 31306 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 10:36:17.660482 31306 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 10:36:17.660691 31306 mini_ranger.cc:66] Stopping Ranger...
I20260808 10:36:18.015807 31306 mini_ranger.cc:68] Stopped Ranger
2026-08-08 10:36:18.016 UTC [10711] LOG: received smart shutdown request
2026-08-08 10:36:18.020 UTC [10711] LOG: background worker "logical replication launcher" (PID 10720) exited with exit code 1
2026-08-08 10:36:18.021 UTC [10715] LOG: shutting down
2026-08-08 10:36:18.199 UTC [10711] LOG: database system is shut down
2026-08-08T10:36:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53595 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:36:18 dist-test-slave-9gcw krb5kdc[11802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:36:18 dist-test-slave-9gcw krb5kdc[11802](info): set up 2 sockets
Aug 08 10:36:18 dist-test-slave-9gcw krb5kdc[11802](info): commencing operation
krb5kdc: starting...
W20260808 10:36:19.446133 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:36:19 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:36:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:36:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/rangeradmin_127.30.146.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/rangeradmin_127.30.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/rangerlookup_127.30.146.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/rangerlookup_127.30.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/HTTP_127.30.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/HTTP_127.30.146.148@KRBTEST.COM.keytab.
I20260808 10:36:19.624469 31306 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwxv9J_/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/postgres -l logfile start
2026-08-08 10:36:22.418 UTC [11837] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 10:36:22.418 UTC [11837] LOG: listening on IPv4 address "127.30.146.148", port 54391
2026-08-08 10:36:22.422 UTC [11837] LOG: listening on Unix socket "/tmp/.s.PGSQL.54391"
2026-08-08 10:36:22.463 UTC [11841] LOG: database system was shut down at 2026-08-08 10:36:20 UTC
2026-08-08 10:36:22.468 UTC [11837] LOG: database system is ready to accept connections
I20260808 10:36:23.580775 31306 mini_postgres.cc:96] Postgres bound to 54391
2026-08-08 10:36:23.588 UTC [11850] FATAL: database "slave" does not exist
127.30.146.148:54391 - accepting connections
I20260808 10:36:23.589184 31306 mini_ranger.cc:161] Starting Ranger...
I20260808 10:36:23.607702 31306 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 10:36:24.155755 31306 mini_ranger.cc:87] Created ranger Postgres database
I20260808 10:36:24.155862 31306 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-admin
2026-08-08 10:36:24,462 [I] DB FLAVOR :POSTGRES
2026-08-08 10:36:24,463 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:36:24,463 [I] Checking connection..
2026-08-08 10:36:24,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 10:36:24,726 [I] Checking connection passed.
2026-08-08 10:36:24,727 [I] --------- Verifying version history table ---------
2026-08-08 10:36:24,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:36:25,011 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:36:25,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:36:25,302 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:36:25,302 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 10:36:25,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 10:36:25.590 UTC [11925] WARNING: there is no transaction in progress
2026-08-08 10:36:25,598 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 10:36:25,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:36:25,878 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 10:36:25,878 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:36:25,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 10:36:26,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 10:36:26,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9gcw.c.gcp-upstream.internal','N') ;"
2026-08-08 10:36:26,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:36:27,026 [I] Table x_portal_user does not exist in database ranger
2026-08-08 10:36:27,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:36:27,315 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 10:36:27,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:36:27,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:36:27,853 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 10:36:27,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 10:36:28.943 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:28.961 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:28.980 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:28.998 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.019 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.037 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.099 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.106 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.123 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.134 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.145 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.161 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.172 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.181 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.188 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29.197 UTC [12040] WARNING: there is no transaction in progress
2026-08-08 10:36:29,944 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 10:36:29,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 10:36:30,196 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 10:36:30,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:36:30,460 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 10:36:30,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:36:30,748 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 10:36:30,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:36:31,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:36:31,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9gcw.c.gcp-upstream.internal';"
2026-08-08 10:36:31,527 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 10:36:31,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 10:36:31,806 [I] DB_PATCHES have already been applied
I20260808 10:36:31.812502 31306 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwxv9J_/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 10:36:31.812602 31306 mini_ranger.cc:193] Using host: 127.30.146.148
I20260808 10:36:31.817088 31306 mini_ranger.cc:227] Ranger admin URL: http://127.30.146.148:36835
Aug 08, 2026 10:36:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:36:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:36:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:36:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:36:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:36:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:36:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 10:36:33.081825 31306 mini_ranger.cc:160] Time spent starting Ranger: real 9.493s user 0.001s sys 0.012s
Aug 08, 2026 10:36:53 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:36:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 10:36:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:36:53 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:36:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 10:36:55.158689 31306 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/rangerkms_127.30.146.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/rangerkms_127.30.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.146.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.146.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.146.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/HTTP_127.30.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.146.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/HTTP_127.30.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 10:36:55 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185415, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 10:36:55.258796 31306 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 10:36:55.269928 31306 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 10:36:56.255682 31306 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 10:36:56.255772 31306 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-kms
I20260808 10:36:56.255795 31306 mini_ranger_kms.cc:227] Using postgres at 127.30.146.148:54391
2026-08-08 10:36:56,444 [I] DB FLAVOR :POSTGRES
2026-08-08 10:36:56,444 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:36:56,445 [I] Checking connection
2026-08-08 10:36:56,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 10:36:56,687 [I] connection success
2026-08-08 10:36:56,688 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 10:36:56,688 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 10:36:56,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 10:36:56,954 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 10:36:56,954 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:36:56,955 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 10:36:56,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.146.148:54391/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 10:36:57,252 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 10:36:57.760540 31306 mini_ranger_kms.cc:310] Created kms service
I20260808 10:36:57.947590 31306 mini_ranger_kms.cc:326] Created kudu user
I20260808 10:36:58.000748 31306 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 10:36:58 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185415, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:36:58.426486 31306 mini_ranger_kms.cc:384] Added ranger policy
I20260808 10:36:58.426638 31306 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwxv9J_/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwxv9J_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 10:36:58.426662 31306 mini_ranger_kms.cc:240] Using host: 127.30.146.148
I20260808 10:36:58.431277 31306 mini_ranger_kms.cc:276] Ranger KMS PID: 12242
I20260808 10:36:58.431372 31306 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.146.148:46767
Aug 08, 2026 10:36:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:36:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:36:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:36:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:37:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:37:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 10:37:00.831612 31306 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.573s user 0.026s sys 0.148s
I20260808 10:37:00.831775 31306 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 10:37:00.831861 31306 mini_ranger_kms.cc:401] 127.30.146.148:46767/kms/v1/keys
Aug 08 10:37:00 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185420, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.146.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 10:37:01 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185420, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.146.148@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
Aug 08 10:37:01 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185420, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.146.148@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
Aug 08 10:37:04 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:37:04.117571 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:36561
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:36561
--ranger_config_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:46767/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:04.241843 12299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:04.242131 12299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:04.242177 12299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:04.246364 12299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:04.246433 12299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:04.246456 12299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:04.246476 12299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:04.246537 12299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:04.252406 12299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.146.148:46767/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:36561
--ranger_config_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:36561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:04.253618 12299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:04.254711 12299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:04.263968 12305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:04.264004 12304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:04.264027 12307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:04.264171 12299 server_base.cc:1034] running on GCE node
I20260808 10:37:04.264971 12299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:04.266306 12299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:04.267478 12299 hybrid_clock.cc:648] HybridClock initialized: now 1786185424267448 us; error 44 us; skew 500 ppm
Aug 08 10:37:04 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185424, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:04.276191 12299 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:37:04.277295 12299 webserver.cc:460] Webserver started at http://127.30.146.190:39757/ using document root <none> and password file <none>
I20260808 10:37:04.277915 12299 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:04.277974 12299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:04.278241 12299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:37:04 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185424, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:04.336427 12299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "f25ca01b923648679b613affeebb967b"
format_stamp: "Formatted at 2026-08-08 10:37:04 on dist-test-slave-9gcw"
server_key: "44171549ca6131a7c0b9ab354d5b5580"
server_key_iv: "1cfe964f898789a096b94df1851f6293"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:04.337287 12299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "f25ca01b923648679b613affeebb967b"
format_stamp: "Formatted at 2026-08-08 10:37:04 on dist-test-slave-9gcw"
server_key: "44171549ca6131a7c0b9ab354d5b5580"
server_key_iv: "1cfe964f898789a096b94df1851f6293"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:04.342828 12299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 10:37:04.360832 12314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:04.362411 12299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:37:04.362574 12299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "f25ca01b923648679b613affeebb967b"
format_stamp: "Formatted at 2026-08-08 10:37:04 on dist-test-slave-9gcw"
server_key: "44171549ca6131a7c0b9ab354d5b5580"
server_key_iv: "1cfe964f898789a096b94df1851f6293"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:04.362733 12299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:04.392844 12299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:04.396070 12299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:04.396306 12299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:04.405055 12299 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:36561
I20260808 10:37:04.405079 12372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:36561 every 8 connection(s)
I20260808 10:37:04.406134 12299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:37:04.408176 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 12299
I20260808 10:37:04.408308 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:37:04.408622 31306 external_mini_cluster.cc:1428] Setting key 6e3d3f63e04b1b8dea93811f67717faa
I20260808 10:37:04.409369 12373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:04.416484 12373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: Bootstrap starting.
I20260808 10:37:04.419085 12373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: Neither blocks nor log segments found. Creating new log.
Aug 08 10:37:04 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:04.420054 12373 log.cc:824] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:04.422520 12373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: No bootstrap required, opened a new log
I20260808 10:37:04.426038 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:04.410183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.30.146.190:36561)
0808 10:37:04.410654 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:04.410683 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:04.410727 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:37:04.411617 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:04.411640 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:04.412069 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:04.412447 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:04.413369 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:04.414858 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:04.416116 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:04.416809 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:04.416964 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:04.420160 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:04.420166 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:04.420184 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:04.420218 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:04.423075 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:04.423555 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:04.423561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:04.423566 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:04.423671 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:04.423970 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:04.423974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:04.423976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:04.424386 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:04.424618 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:04.424891 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 10:37:04.425231 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":151,"threads_started":1}
I20260808 10:37:04.426716 12373 raft_consensus.cc:348] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } }
I20260808 10:37:04.426990 12373 raft_consensus.cc:374] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:04.427078 12373 raft_consensus.cc:729] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25ca01b923648679b613affeebb967b, State: Initialized, Role: FOLLOWER
I20260808 10:37:04.427560 12373 consensus_queue.cc:260] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } }
I20260808 10:37:04.427750 12373 raft_consensus.cc:388] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:37:04.427857 12373 raft_consensus.cc:482] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:37:04.427960 12373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:04.429132 12373 raft_consensus.cc:504] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } }
I20260808 10:37:04.429502 12373 leader_election.cc:302] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f25ca01b923648679b613affeebb967b; no voters:
I20260808 10:37:04.429858 12373 leader_election.cc:288] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:37:04.429991 12378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:37:04.430279 12378 raft_consensus.cc:686] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 1 LEADER]: Becoming Leader. State: Replica: f25ca01b923648679b613affeebb967b, State: Running, Role: LEADER
I20260808 10:37:04.430753 12378 consensus_queue.cc:237] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } }
I20260808 10:37:04.431154 12373 sys_catalog.cc:564] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:04.432210 12378 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f25ca01b923648679b613affeebb967b. Latest consensus state: current_term: 1 leader_uuid: "f25ca01b923648679b613affeebb967b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } } }
I20260808 10:37:04.432571 12378 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:04.432993 12387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:37:04.433414 12380 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f25ca01b923648679b613affeebb967b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } } }
I20260808 10:37:04.433509 12380 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:04.436376 12387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:37:04.436818 12373 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:37:04.455055 12387 catalog_manager.cc:1347] Generated new cluster ID: 41883cc4809e4f779d5bd3680add8644
I20260808 10:37:04.455150 12387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:37:04.474305 12387 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:37:04.475739 12387 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:37:04.496553 12387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: Generated new TSK 0
I20260808 10:37:04.497655 12387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:37:05.245298 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:46767/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:36561
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:05.372123 12422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:05.372454 12422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:05.372547 12422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:05.376967 12422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:05.377050 12422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:05.377190 12422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:37:05.383101 12422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.146.148:46767/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:36561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:05.384320 12422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:05.385761 12422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:05.398452 12432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:05.398494 12422 server_base.cc:1034] running on GCE node
W20260808 10:37:05.398589 12429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:05.398684 12430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:05.399251 12422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:05.399878 12422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:05.401082 12422 hybrid_clock.cc:648] HybridClock initialized: now 1786185425401054 us; error 54 us; skew 500 ppm
Aug 08 10:37:05 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185425, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:05.404510 12422 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:37:05.405870 12422 webserver.cc:460] Webserver started at http://127.30.146.129:39225/ using document root <none> and password file <none>
I20260808 10:37:05.406642 12422 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:05.406809 12422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:05.407100 12422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:37:05 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185425, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:05.432089 12422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "91e8fe54c1bc4a96b331e21d0c0c735f"
format_stamp: "Formatted at 2026-08-08 10:37:05 on dist-test-slave-9gcw"
server_key: "e99acb18c95dd726b7661113b1c0d455"
server_key_iv: "3d5cd0b9e0dfc1e9cd75ec2bc9187e6f"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:05.433216 12422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "91e8fe54c1bc4a96b331e21d0c0c735f"
format_stamp: "Formatted at 2026-08-08 10:37:05 on dist-test-slave-9gcw"
server_key: "e99acb18c95dd726b7661113b1c0d455"
server_key_iv: "3d5cd0b9e0dfc1e9cd75ec2bc9187e6f"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:05.437672 12422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:37:05.454198 12441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:05.455668 12422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:37:05.455821 12422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "91e8fe54c1bc4a96b331e21d0c0c735f"
format_stamp: "Formatted at 2026-08-08 10:37:05 on dist-test-slave-9gcw"
server_key: "e99acb18c95dd726b7661113b1c0d455"
server_key_iv: "3d5cd0b9e0dfc1e9cd75ec2bc9187e6f"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:05.455973 12422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:05.470953 12422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:05.474943 12422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:05.475206 12422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:05.476061 12422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:05.477223 12422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:05.477329 12422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:05.477461 12422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:05.477555 12422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:05.491384 12422 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:44825
I20260808 10:37:05.494556 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 12422
I20260808 10:37:05.494740 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:37:05.495069 31306 external_mini_cluster.cc:1428] Setting key c3b0e132e377fd0c9d4c3b399beafe7f
I20260808 10:37:05.492743 12422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:37:05.497198 12565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:44825 every 8 connection(s)
Aug 08 10:37:05 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185425, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:05.517294 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:05.501364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:42087 (local address 127.30.146.190:36561)
0808 10:37:05.502139 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:05.502144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:05.503880 (+ 1736us) server_negotiation.cc:407] Connection header received
0808 10:37:05.503933 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:05.503936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:05.503993 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:05.504084 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:05.505815 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:05.506648 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:05.508503 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:05.509049 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:05.509178 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:05.512513 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:05.512517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:05.512520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:05.512551 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:05.514636 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:05.515293 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:05.515297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:05.515299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:05.515416 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:05.515848 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:05.515851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:05.515853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:05.516087 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:05.516174 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:05.516891 (+ 717us) server_negotiation.cc:299] Negotiation successful
0808 10:37:05.517030 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":98,"threads_started":1}
I20260808 10:37:05.519532 12569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:05.500310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:36561 (local address 127.30.146.129:42087)
0808 10:37:05.502409 (+ 2099us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:05.502466 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:05.503558 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:05.504406 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:05.504415 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:05.504850 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:05.505610 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:05.505631 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:05.506846 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:05.506850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:05.508357 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:05.508371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:05.509409 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:05.509413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:05.509650 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:05.510434 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:05.510461 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:05.512299 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:05.514804 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:05.514811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:05.514829 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:05.515174 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:05.515530 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:05.515533 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:05.515535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:05.515729 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:05.516245 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:05.516252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:05.516579 (+ 327us) client_negotiation.cc:769] Sending connection context
0808 10:37:05.517864 (+ 1285us) client_negotiation.cc:240] Negotiation successful
0808 10:37:05.518255 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1875,"thread_start_us":152,"threads_started":1}
I20260808 10:37:05.521399 12566 heartbeater.cc:347] Connected to a master server at 127.30.146.190:36561
I20260808 10:37:05.521685 12566 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:37:05.522318 12566 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
I20260808 10:37:05.524101 12332 ts_manager.cc:194] Registered new tserver with Master: 91e8fe54c1bc4a96b331e21d0c0c735f (127.30.146.129:44825)
I20260808 10:37:05.525628 12332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:42087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:37:05.549605 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:46767/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:36561
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:37:05 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185425, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 10:37:05.710495 12574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:05.711048 12574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:05.711180 12574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:05.718211 12574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:05.718312 12574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:05.718416 12574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:37:05.727631 12574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.146.148:46767/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:36561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:05.729408 12574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:05.730873 12574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:05.744761 12582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:05.744746 12581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:05.744943 12584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:05.746840 12574 server_base.cc:1034] running on GCE node
I20260808 10:37:05.747409 12574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:05.748339 12574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:05.749645 12574 hybrid_clock.cc:648] HybridClock initialized: now 1786185425749569 us; error 92 us; skew 500 ppm
Aug 08 10:37:05 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185425, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:05.753756 12574 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:37:05.755412 12574 webserver.cc:460] Webserver started at http://127.30.146.130:43881/ using document root <none> and password file <none>
I20260808 10:37:05.756377 12574 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:05.756570 12574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:05.756959 12574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:37:05 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185425, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:05.782011 12574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "21406271d4724f72a848035e9efbdf93"
format_stamp: "Formatted at 2026-08-08 10:37:05 on dist-test-slave-9gcw"
server_key: "4f8480755721992cd3fff3b73b3108be"
server_key_iv: "9490af2eb1ce9b1699e73ced19d3e797"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:05.783492 12574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "21406271d4724f72a848035e9efbdf93"
format_stamp: "Formatted at 2026-08-08 10:37:05 on dist-test-slave-9gcw"
server_key: "4f8480755721992cd3fff3b73b3108be"
server_key_iv: "9490af2eb1ce9b1699e73ced19d3e797"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:05.788815 12574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 10:37:05.818742 12591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:05.822245 12574 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.000s
I20260808 10:37:05.822443 12574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "21406271d4724f72a848035e9efbdf93"
format_stamp: "Formatted at 2026-08-08 10:37:05 on dist-test-slave-9gcw"
server_key: "4f8480755721992cd3fff3b73b3108be"
server_key_iv: "9490af2eb1ce9b1699e73ced19d3e797"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:05.822723 12574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:05.877725 12574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:05.881711 12574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:05.881994 12574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:05.882818 12574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:05.884006 12574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:05.884123 12574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:05.884246 12574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:05.884318 12574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:05.899150 12574 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:39569
I20260808 10:37:05.900301 12712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:39569 every 8 connection(s)
I20260808 10:37:05.900352 12574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:37:05.905470 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 12574
I20260808 10:37:05.905628 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:37:05.905941 31306 external_mini_cluster.cc:1428] Setting key 65aeaa5f7d0bb306f9d5d99d111b2294
Aug 08 10:37:05 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185425, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:05.919780 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:05.903293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:40361 (local address 127.30.146.190:36561)
0808 10:37:05.903452 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:05.903458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:05.904519 (+ 1061us) server_negotiation.cc:407] Connection header received
0808 10:37:05.905827 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:05.905832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:05.905892 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:05.905951 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:05.908212 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:05.908967 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:05.910712 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:05.911085 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:05.911219 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:05.914783 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:05.914788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:05.914791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:05.914822 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:05.916891 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:05.917625 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:05.917629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:05.917631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:05.917737 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:05.918244 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:05.918247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:05.918249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:05.918503 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:05.918597 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:05.919432 (+ 835us) server_negotiation.cc:299] Negotiation successful
0808 10:37:05.919586 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:37:05.921967 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:05.903751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:36561 (local address 127.30.146.130:40361)
0808 10:37:05.904352 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:05.904411 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:05.905560 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:05.906636 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:05.906650 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:05.907025 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:05.908017 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:05.908039 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:05.909135 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:05.909181 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:05.910563 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:05.910574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:05.911415 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:05.911420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:05.911642 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:05.912508 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:05.912532 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:05.914549 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:05.917043 (+ 2494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:05.917049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:05.917068 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:05.917486 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:05.917882 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:05.917886 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:05.917889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:05.918101 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:05.918708 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:05.918718 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:05.919096 (+ 378us) client_negotiation.cc:769] Sending connection context
0808 10:37:05.920317 (+ 1221us) client_negotiation.cc:240] Negotiation successful
0808 10:37:05.920738 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":154,"threads_started":1}
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:37:05.923929 12713 heartbeater.cc:347] Connected to a master server at 127.30.146.190:36561
I20260808 10:37:05.924260 12713 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:05.924997 12713 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
I20260808 10:37:05.926456 12332 ts_manager.cc:194] Registered new tserver with Master: 21406271d4724f72a848035e9efbdf93 (127.30.146.130:39569)
I20260808 10:37:05.927317 12332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:40361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:37:05.958298 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:46767/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:36561
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:06.137663 12719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:06.138526 12719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:06.138679 12719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:06.145447 12719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:06.145651 12719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:06.145821 12719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:37:06.154124 12719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.146.148:46767/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:36561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:06.155838 12719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:06.157521 12719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 08 10:37:06 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185425, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:06.171866 12719 server_base.cc:1034] running on GCE node
W20260808 10:37:06.172091 12725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:06.172715 12729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:06.173398 12726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:06.173993 12719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:06.174803 12719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:06.176004 12719 hybrid_clock.cc:648] HybridClock initialized: now 1786185426175921 us; error 97 us; skew 500 ppm
Aug 08 10:37:06 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185426, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:06.179925 12719 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:37:06.181108 12719 webserver.cc:460] Webserver started at http://127.30.146.131:44327/ using document root <none> and password file <none>
I20260808 10:37:06.181902 12719 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:06.182060 12719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:06.182381 12719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:37:06 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185426, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:06.199039 12719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a"
format_stamp: "Formatted at 2026-08-08 10:37:06 on dist-test-slave-9gcw"
server_key: "4194e0290db631ddad4633a446f1d8dc"
server_key_iv: "3a058c576b47f4fdabe949c91ec9d2fc"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:06.200953 12719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a"
format_stamp: "Formatted at 2026-08-08 10:37:06 on dist-test-slave-9gcw"
server_key: "4194e0290db631ddad4633a446f1d8dc"
server_key_iv: "3a058c576b47f4fdabe949c91ec9d2fc"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:06.206871 12719 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 10:37:06.230544 12737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:06.232321 12719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:37:06.232465 12719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a"
format_stamp: "Formatted at 2026-08-08 10:37:06 on dist-test-slave-9gcw"
server_key: "4194e0290db631ddad4633a446f1d8dc"
server_key_iv: "3a058c576b47f4fdabe949c91ec9d2fc"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:06.232594 12719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:06.284721 12719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:06.288560 12719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:06.288918 12719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:06.289685 12719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:06.290807 12719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:06.290928 12719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:06.291030 12719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:06.291100 12719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:06.304747 12719 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:46581
I20260808 10:37:06.306049 12719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:37:06.306948 12865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:46581 every 8 connection(s)
I20260808 10:37:06.312723 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 12719
I20260808 10:37:06.312858 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:37:06.313169 31306 external_mini_cluster.cc:1428] Setting key 6bbeca03279c1bf7876c198e6cdbf2f6
Aug 08 10:37:06 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185426, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:06.357309 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:06.310751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:60751 (local address 127.30.146.190:36561)
0808 10:37:06.310891 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:06.310895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:06.322134 (+ 11239us) server_negotiation.cc:407] Connection header received
0808 10:37:06.323473 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:06.323476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:06.323535 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:06.323590 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:06.325400 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:06.326273 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:06.335277 (+ 9004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:06.335683 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:06.335779 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:06.351689 (+ 15910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:06.351694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:06.351696 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:06.351730 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:06.353793 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:06.355300 (+ 1507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:06.355304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:06.355306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:06.355421 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:06.355879 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:06.355882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:06.355884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:06.356139 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:06.356220 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:06.356962 (+ 742us) server_negotiation.cc:299] Negotiation successful
0808 10:37:06.357106 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:37:06.359458 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:06.309613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:36561 (local address 127.30.146.131:60751)
0808 10:37:06.321902 (+ 12289us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:06.321990 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:06.323203 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:06.324063 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:06.324073 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:06.324482 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:06.325220 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:06.325242 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:06.328085 (+ 2843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:06.328088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:06.329604 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:06.335093 (+ 5489us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:06.336020 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:06.336024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:06.336258 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:06.349371 (+ 13113us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:06.349400 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:06.351417 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:06.354743 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:06.354751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:06.354783 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:06.355171 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:06.355536 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:06.355538 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:06.355540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:06.355747 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:06.356292 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:06.356299 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:06.356658 (+ 359us) client_negotiation.cc:769] Sending connection context
0808 10:37:06.357946 (+ 1288us) client_negotiation.cc:240] Negotiation successful
0808 10:37:06.358343 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11928,"thread_start_us":156,"threads_started":1}
I20260808 10:37:06.361027 12866 heartbeater.cc:347] Connected to a master server at 127.30.146.190:36561
I20260808 10:37:06.361306 12866 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:06.361915 12866 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
I20260808 10:37:06.363099 12332 ts_manager.cc:194] Registered new tserver with Master: 1c9a0a6494b84c7d9c22bd31b5d4a79a (127.30.146.131:46581)
I20260808 10:37:06.363850 12332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:60751
I20260808 10:37:06.367750 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:37:06.368643 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 12422
I20260808 10:37:06.394189 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 12574
I20260808 10:37:06.405483 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 12719
I20260808 10:37:06.421427 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 12299
I20260808 10:37:06.432174 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:36561
--webserver_interface=127.30.146.190
--webserver_port=39757
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:36561
--ranger_config_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:46767/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.146.148:46767/kms with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:06.549413 12869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:06.549679 12869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:06.549747 12869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:06.553853 12869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:06.553911 12869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:06.553930 12869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:06.553968 12869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:06.554003 12869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:06.559547 12869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.146.148:46767/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:36561
--ranger_config_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:36561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=39757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:06.560640 12869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:06.561758 12869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:06.571828 12874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:06.571828 12877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:06.571826 12875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:06.572414 12869 server_base.cc:1034] running on GCE node
I20260808 10:37:06.573069 12869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:06.574491 12869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:06.575685 12869 hybrid_clock.cc:648] HybridClock initialized: now 1786185426575663 us; error 35 us; skew 500 ppm
Aug 08 10:37:06 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185426, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:06.579144 12869 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:37:06.580240 12869 webserver.cc:460] Webserver started at http://127.30.146.190:39757/ using document root <none> and password file <none>
I20260808 10:37:06.580893 12869 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:06.580981 12869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:37:06 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185426, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:06.630144 12869 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:37:06.632633 12887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:06.634068 12869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:06.634238 12869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "f25ca01b923648679b613affeebb967b"
format_stamp: "Formatted at 2026-08-08 10:37:04 on dist-test-slave-9gcw"
server_key: "44171549ca6131a7c0b9ab354d5b5580"
server_key_iv: "1cfe964f898789a096b94df1851f6293"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:06.634824 12869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:06.645435 12869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:06.648552 12869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:06.648780 12869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:06.656431 12869 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:36561
I20260808 10:37:06.656459 12945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:36561 every 8 connection(s)
I20260808 10:37:06.657595 12869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:37:06.660276 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 12869
I20260808 10:37:06.660808 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:46767/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:44825
--local_ip_for_outbound_sockets=127.30.146.129
--tserver_master_addrs=127.30.146.190:36561
--webserver_port=39225
--webserver_interface=127.30.146.129
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.146.148:46767/kms with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:37:06.663066 12946 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:37:06.665261 12946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: Bootstrap starting.
I20260808 10:37:06.687466 12946 log.cc:824] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:06.696421 12946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:37:06.696911 12946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: Bootstrap complete.
I20260808 10:37:06.702867 12946 raft_consensus.cc:348] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } }
I20260808 10:37:06.703431 12946 raft_consensus.cc:729] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25ca01b923648679b613affeebb967b, State: Initialized, Role: FOLLOWER
I20260808 10:37:06.704041 12946 consensus_queue.cc:260] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } }
I20260808 10:37:06.704214 12946 raft_consensus.cc:388] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:37:06.704330 12946 raft_consensus.cc:482] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:37:06.704442 12946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:37:06.707111 12946 raft_consensus.cc:504] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } }
I20260808 10:37:06.707484 12946 leader_election.cc:302] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f25ca01b923648679b613affeebb967b; no voters:
I20260808 10:37:06.707842 12946 leader_election.cc:288] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:37:06.707983 12950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:37:06.708312 12950 raft_consensus.cc:686] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [term 2 LEADER]: Becoming Leader. State: Replica: f25ca01b923648679b613affeebb967b, State: Running, Role: LEADER
I20260808 10:37:06.708669 12950 consensus_queue.cc:237] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } }
I20260808 10:37:06.709221 12946 sys_catalog.cc:564] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:06.709748 12950 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f25ca01b923648679b613affeebb967b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } } }
I20260808 10:37:06.709939 12950 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:06.710196 12950 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f25ca01b923648679b613affeebb967b. Latest consensus state: current_term: 2 leader_uuid: "f25ca01b923648679b613affeebb967b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25ca01b923648679b613affeebb967b" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 36561 } } }
I20260808 10:37:06.710327 12950 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:06.710623 12957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:37:06.715410 12957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:37:06.715974 12946 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:37:06.718068 12957 catalog_manager.cc:1259] Loaded cluster ID: 41883cc4809e4f779d5bd3680add8644
I20260808 10:37:06.718199 12957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:37:06.720722 12957 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:37:06.721606 12957 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: Loaded TSK: 0
I20260808 10:37:06.722879 12957 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:37:06.814613 12948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:06.815179 12948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:06.815358 12948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:06.822603 12948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:06.822774 12948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:06.822942 12948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:37:06.831741 12948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.146.148:46767/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:44825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=39225
--tserver_master_addrs=127.30.146.190:36561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:06.833487 12948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:06.834946 12948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:06.849472 12987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:06.849805 12948 server_base.cc:1034] running on GCE node
W20260808 10:37:06.850220 12986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:06.850546 12989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:06.851135 12948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:06.851972 12948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:06.853132 12948 hybrid_clock.cc:648] HybridClock initialized: now 1786185426853070 us; error 78 us; skew 500 ppm
Aug 08 10:37:06 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185426, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:06.857136 12948 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:37:06.858409 12948 webserver.cc:460] Webserver started at http://127.30.146.129:39225/ using document root <none> and password file <none>
I20260808 10:37:06.859225 12948 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:06.859401 12948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:37:06 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185426, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:06.893501 12948 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.001s
I20260808 10:37:06.897330 12997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:06.899067 12948 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:37:06.899201 12948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "91e8fe54c1bc4a96b331e21d0c0c735f"
format_stamp: "Formatted at 2026-08-08 10:37:05 on dist-test-slave-9gcw"
server_key: "e99acb18c95dd726b7661113b1c0d455"
server_key_iv: "3d5cd0b9e0dfc1e9cd75ec2bc9187e6f"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:06.899988 12948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:06.926112 12948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:06.930192 12948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:06.930473 12948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:06.931203 12948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:06.932345 12948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:06.932454 12948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:06.932597 12948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:06.932704 12948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:06.944031 12948 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:44825
I20260808 10:37:06.944361 13123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:44825 every 8 connection(s)
I20260808 10:37:06.949018 12948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:37:06.958425 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 12948
I20260808 10:37:06.959007 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:46767/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:39569
--local_ip_for_outbound_sockets=127.30.146.130
--tserver_master_addrs=127.30.146.190:36561
--webserver_port=43881
--webserver_interface=127.30.146.130
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.146.148:46767/kms with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:37:06 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185426, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:06.979465 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:06.953305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:53535 (local address 127.30.146.190:36561)
0808 10:37:06.956961 (+ 3656us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:06.956991 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:06.957047 (+ 56us) server_negotiation.cc:407] Connection header received
0808 10:37:06.957410 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:06.957426 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:06.957906 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:06.958320 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:06.960205 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:06.961885 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:06.963926 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:06.964981 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:06.965256 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:06.968881 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:06.968888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:06.968909 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:06.968949 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:06.971384 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:06.972772 (+ 1388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:06.972780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:06.972787 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:06.973334 (+ 547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:06.973857 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:06.973861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:06.973862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:06.974471 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:06.974612 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:06.978028 (+ 3416us) server_negotiation.cc:299] Negotiation successful
0808 10:37:06.978389 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3456,"thread_start_us":133,"threads_started":1}
I20260808 10:37:06.980383 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:06.951896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:36561 (local address 127.30.146.129:53535)
0808 10:37:06.952541 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:06.952605 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:06.953757 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:06.958879 (+ 5122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:06.958890 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:06.959284 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:06.959968 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:06.959992 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:06.962076 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:06.962080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:06.963531 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:06.963545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:06.965534 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:06.965539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:06.965765 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:06.966520 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:06.966545 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:06.968570 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:06.971564 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:06.971572 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:06.971590 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:06.972015 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:06.973482 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:06.973486 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:06.973488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:06.973719 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:06.974693 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:06.974701 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:06.975283 (+ 582us) client_negotiation.cc:769] Sending connection context
0808 10:37:06.975557 (+ 274us) client_negotiation.cc:240] Negotiation successful
0808 10:37:06.976262 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":180,"threads_started":1}
I20260808 10:37:06.984069 13124 heartbeater.cc:347] Connected to a master server at 127.30.146.190:36561
I20260808 10:37:06.984324 13124 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:06.984961 13124 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
W20260808 10:37:06.987484 13124 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:36561 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:37:06.987632 13124 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:06.987840 13124 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
I20260808 10:37:06.990404 13124 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:06.990645 13124 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
W20260808 10:37:06.991616 13124 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:37:07.141637 13128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:07.142129 13128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:07.142329 13128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:07.148729 13128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:07.148926 13128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:07.149127 13128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:37:07.157732 13128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.146.148:46767/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:39569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=43881
--tserver_master_addrs=127.30.146.190:36561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:07.159476 13128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:07.160938 13128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:07.175276 13135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:07.175282 13137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:07.175985 13134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:07.176604 13128 server_base.cc:1034] running on GCE node
I20260808 10:37:07.177300 13128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:07.178161 13128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:07.179347 13128 hybrid_clock.cc:648] HybridClock initialized: now 1786185427179307 us; error 57 us; skew 500 ppm
Aug 08 10:37:07 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185427, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:07.183183 13128 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:37:07.184607 13128 webserver.cc:460] Webserver started at http://127.30.146.130:43881/ using document root <none> and password file <none>
I20260808 10:37:07.185437 13128 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:07.185563 13128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:37:07 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185427, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:07.207858 13128 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:37:07.213398 13146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:07.215277 13128 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260808 10:37:07.215534 13128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "21406271d4724f72a848035e9efbdf93"
format_stamp: "Formatted at 2026-08-08 10:37:05 on dist-test-slave-9gcw"
server_key: "4f8480755721992cd3fff3b73b3108be"
server_key_iv: "9490af2eb1ce9b1699e73ced19d3e797"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:07.216383 13128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:07.234127 13128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:07.237951 13128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:07.238191 13128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:07.238970 13128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:07.240135 13128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:07.240198 13128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:07.240253 13128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:07.240284 13128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:07.251480 13128 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:39569
I20260808 10:37:07.252862 13128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:37:07.254194 13269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:39569 every 8 connection(s)
I20260808 10:37:07.253093 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 13128
I20260808 10:37:07.255620 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.146.148:46767/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:46581
--local_ip_for_outbound_sockets=127.30.146.131
--tserver_master_addrs=127.30.146.190:36561
--webserver_port=44327
--webserver_interface=127.30.146.131
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.146.148:46767/kms with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:37:07 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185427, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:07.272231 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:07.255718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:57985 (local address 127.30.146.190:36561)
0808 10:37:07.255860 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:07.255865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:07.256862 (+ 997us) server_negotiation.cc:407] Connection header received
0808 10:37:07.258181 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:07.258185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:07.258245 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:07.258302 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:07.260035 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:07.260881 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:07.262706 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:07.263091 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:07.263183 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:07.267041 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:07.267046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:07.267049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:07.267113 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:07.269243 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:07.270027 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:07.270031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:07.270032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:07.270154 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:07.270643 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:07.270647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:07.270649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:07.270911 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:07.271005 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:07.271843 (+ 838us) server_negotiation.cc:299] Negotiation successful
0808 10:37:07.272014 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:37:07.274366 13272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:07.256211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:36561 (local address 127.30.146.130:57985)
0808 10:37:07.256707 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:07.256759 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:07.257907 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:07.258748 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:07.258757 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:07.259136 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:07.259843 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:07.259867 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:07.261060 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:07.261063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:07.262537 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:07.262548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:07.263411 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:07.263416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:07.263638 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:07.264545 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:07.264569 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:07.266760 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:07.269424 (+ 2664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:07.269432 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:07.269451 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:07.269877 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:07.270293 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:07.270297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:07.270299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:07.270505 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:07.271098 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:07.271106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:07.271478 (+ 372us) client_negotiation.cc:769] Sending connection context
0808 10:37:07.272766 (+ 1288us) client_negotiation.cc:240] Negotiation successful
0808 10:37:07.273191 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":147,"threads_started":1}
I20260808 10:37:07.276175 13270 heartbeater.cc:347] Connected to a master server at 127.30.146.190:36561
I20260808 10:37:07.276516 13270 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:07.277266 13270 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
W20260808 10:37:07.279181 13270 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:36561 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:37:07.279390 13270 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:07.279654 13270 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
I20260808 10:37:07.280661 13270 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:07.280905 13270 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
W20260808 10:37:07.281827 13270 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:37:07.411996 13273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:07.412302 13273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:07.412431 13273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:07.417742 13273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:07.417893 13273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:07.418037 13273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:37:07.424721 13273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.146.148:46767/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:46581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=44327
--tserver_master_addrs=127.30.146.190:36561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:07.425906 13273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:07.427204 13273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:07.438833 13283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:07.439122 13273 server_base.cc:1034] running on GCE node
W20260808 10:37:07.438740 13280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:07.438932 13281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:07.439929 13273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:07.440793 13273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:07.442067 13273 hybrid_clock.cc:648] HybridClock initialized: now 1786185427442025 us; error 63 us; skew 500 ppm
Aug 08 10:37:07 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185427, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:07.445910 13273 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:37:07.447113 13273 webserver.cc:460] Webserver started at http://127.30.146.131:44327/ using document root <none> and password file <none>
I20260808 10:37:07.447911 13273 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:07.448086 13273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:37:07 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185427, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:07.478652 13273 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:37:07.481962 13291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:07.483626 13273 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:37:07.483778 13273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a"
format_stamp: "Formatted at 2026-08-08 10:37:06 on dist-test-slave-9gcw"
server_key: "4194e0290db631ddad4633a446f1d8dc"
server_key_iv: "3a058c576b47f4fdabe949c91ec9d2fc"
server_key_version: "kuduclusterkey@0"
I20260808 10:37:07.484519 13273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:07.498646 13273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:07.502420 13273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:07.502687 13273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:07.503423 13273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:07.504431 13273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:07.504501 13273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:07.504572 13273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:07.504602 13273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:07.516156 13273 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:46581
I20260808 10:37:07.516237 13418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:46581 every 8 connection(s)
I20260808 10:37:07.517632 13273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:37:07.527107 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 13273
I20260808 10:37:07.540416 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:07.528309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47328 (local address 127.30.146.190:36561)
0808 10:37:07.528628 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:07.528639 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:07.530592 (+ 1953us) server_negotiation.cc:407] Connection header received
0808 10:37:07.530782 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:07.530788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:07.530848 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:07.530947 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:07.531735 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:07.532532 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:07.533560 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:07.533892 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:07.534044 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:07.535806 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:07.535810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:07.535813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:07.535840 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:07.537797 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:07.538401 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:07.538405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:07.538406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:07.538495 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:07.538831 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:07.538834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:07.538835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:07.539095 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:07.540002 (+ 907us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:07.540079 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 10:37:07.540218 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":110,"threads_started":1}
Aug 08 10:37:07 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185427, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:07.547607 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:07.526512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:42867 (local address 127.30.146.190:36561)
0808 10:37:07.526650 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:07.526655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:07.532103 (+ 5448us) server_negotiation.cc:407] Connection header received
0808 10:37:07.534930 (+ 2827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:07.534933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:07.534990 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:07.535045 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:07.536942 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:07.537817 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:07.539625 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:07.539967 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:07.540044 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:07.542911 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:07.542915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:07.542919 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:07.542947 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:07.544741 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:07.545562 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:07.545566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:07.545568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:07.545662 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:07.546217 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:07.546220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:07.546222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:07.546462 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:07.546549 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:07.547290 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 10:37:07.547425 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58,"spinlock_wait_cycles":3328}
I20260808 10:37:07.549611 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:07.527069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:36561 (local address 127.30.146.131:42867)
0808 10:37:07.531928 (+ 4859us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:07.532007 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:07.534651 (+ 2644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:07.535550 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:07.535563 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:07.535949 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:07.536747 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:07.536779 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:07.537975 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:07.537978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:07.539487 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:07.539499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:07.540271 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:07.540276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:07.540486 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:07.541203 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:07.541227 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:07.542733 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:07.544958 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:07.544965 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:07.545011 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:07.545420 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:07.545835 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:07.545838 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:07.545840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:07.546075 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:07.546620 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:07.546627 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:07.546956 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 10:37:07.548088 (+ 1132us) client_negotiation.cc:240] Negotiation successful
0808 10:37:07.548477 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3741,"thread_start_us":183,"threads_started":1}
I20260808 10:37:07.551446 13419 heartbeater.cc:347] Connected to a master server at 127.30.146.190:36561
I20260808 10:37:07.551705 13419 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:07.552314 13419 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
W20260808 10:37:07.554103 13419 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:36561 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:37:07.554250 13419 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:07.554421 13419 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
I20260808 10:37:07.555229 13419 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:07.555426 13419 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
W20260808 10:37:07.556198 13419 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:37:07.991880 13124 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:07.992210 13124 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
I20260808 10:37:07.993680 12905 ts_manager.cc:194] Registered new tserver with Master: 91e8fe54c1bc4a96b331e21d0c0c735f (127.30.146.129:44825)
I20260808 10:37:07.995580 12905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:53535
Aug 08 10:37:08 dist-test-slave-9gcw krb5kdc[11802](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185428, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:08.282143 13270 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:08.282519 13270 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
I20260808 10:37:08.283504 12905 ts_manager.cc:194] Registered new tserver with Master: 21406271d4724f72a848035e9efbdf93 (127.30.146.130:39569)
I20260808 10:37:08.284627 12905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:57985
Aug 08 10:37:08 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185428, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:08.556475 13419 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:08.556788 13419 heartbeater.cc:511] Master 127.30.146.190:36561 requested a full tablet report, sending...
I20260808 10:37:08.557624 12905 ts_manager.cc:194] Registered new tserver with Master: 1c9a0a6494b84c7d9c22bd31b5d4a79a (127.30.146.131:46581)
I20260808 10:37:08.558498 12905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:42867
I20260808 10:37:08.564389 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:37:08.575917 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.566173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47330 (local address 127.30.146.190:36561)
0808 10:37:08.566469 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:08.566476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:08.566747 (+ 271us) server_negotiation.cc:407] Connection header received
0808 10:37:08.566918 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:08.566921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:08.566981 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:08.567084 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:08.567854 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.568795 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.569918 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.570206 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.570329 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.571998 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:08.572009 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:08.572012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:08.572038 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:08.573939 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:08.574447 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:08.574451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:08.574453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:08.574559 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:08.574884 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:08.574887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:08.574889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:08.575097 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:08.575273 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:08.575562 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 10:37:08.575715 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":111,"threads_started":1}
I20260808 10:37:08.580250 12905 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:37:08.584262 12905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:37:08.603941 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.595359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:39569 (local address 127.0.0.1:41156)
0808 10:37:08.597743 (+ 2384us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:08.597805 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:08.597942 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:08.598617 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:08.598623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:08.598649 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:08.598812 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:08.598821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.600446 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.600449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:08.601964 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:08.601976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.603370 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.603374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:08.603491 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.603530 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:37:08.603642 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 10:37:08.603774 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2183,"thread_start_us":142,"threads_started":1}
I20260808 10:37:08.604652 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.596288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.30.146.130:39569)
0808 10:37:08.598189 (+ 1901us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:08.598196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:08.598217 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:37:08.598281 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:08.598285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:08.598451 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:08.598629 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:08.598969 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.600303 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.602116 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.603033 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.603155 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.603299 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:08.604358 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0808 10:37:08.604490 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1783,"thread_start_us":118,"threads_started":1}
I20260808 10:37:08.607121 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.596288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59162 (local address 127.30.146.129:44825)
0808 10:37:08.597355 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:08.597364 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:08.599034 (+ 1670us) server_negotiation.cc:407] Connection header received
0808 10:37:08.599376 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:08.599383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:08.599576 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:08.599696 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:08.601493 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.602741 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.605205 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.606188 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.606303 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.606446 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:08.606868 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 10:37:08.606975 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":136,"threads_started":1}
I20260808 10:37:08.607520 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.595484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:44825 (local address 127.0.0.1:59162)
0808 10:37:08.598925 (+ 3441us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:08.598943 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:08.599108 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:08.599882 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:08.599885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:08.599903 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:08.599992 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:08.601355 (+ 1363us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.602901 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.602904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:08.605019 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:08.605032 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.606555 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.606559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:08.606653 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.606677 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:37:08.607286 (+ 609us) client_negotiation.cc:240] Negotiation successful
0808 10:37:08.607374 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3355,"thread_start_us":86,"threads_started":1}
I20260808 10:37:08.608985 13199 tablet_service.cc:1467] Processing CreateTablet for tablet 0a6ed6e537594fceaa49909b9e7dcdc3 (DEFAULT_TABLE table=test-table [id=7a2edca3640b4b1dba7e70eed0c89410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:08.610699 13199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a6ed6e537594fceaa49909b9e7dcdc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:08.611666 13046 tablet_service.cc:1467] Processing CreateTablet for tablet 0a6ed6e537594fceaa49909b9e7dcdc3 (DEFAULT_TABLE table=test-table [id=7a2edca3640b4b1dba7e70eed0c89410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:08.612766 13046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a6ed6e537594fceaa49909b9e7dcdc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:08.616413 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.597133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:46581 (local address 127.0.0.1:59342)
0808 10:37:08.605229 (+ 8096us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:08.605245 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:08.605316 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:08.610688 (+ 5372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:08.610692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:08.610715 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:08.610816 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:08.610823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.612349 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.612352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:08.614466 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:08.614478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.615917 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.615922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:08.616019 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.616050 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 10:37:08.616136 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:37:08.616247 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8036,"thread_start_us":131,"threads_started":1}
I20260808 10:37:08.617007 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.597440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.30.146.131:46581)
0808 10:37:08.609950 (+ 12510us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:08.609957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:08.609986 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:37:08.610069 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:08.610075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:08.610313 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:08.610475 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:08.610971 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.612199 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.614622 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.615558 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.615698 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.615842 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:08.616745 (+ 903us) server_negotiation.cc:299] Negotiation successful
0808 10:37:08.616862 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12377,"thread_start_us":154,"threads_started":1}
I20260808 10:37:08.620339 13346 tablet_service.cc:1467] Processing CreateTablet for tablet 0a6ed6e537594fceaa49909b9e7dcdc3 (DEFAULT_TABLE table=test-table [id=7a2edca3640b4b1dba7e70eed0c89410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:08.621508 13346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a6ed6e537594fceaa49909b9e7dcdc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:08.632318 13455 tablet_bootstrap.cc:492] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: Bootstrap starting.
I20260808 10:37:08.633265 13456 tablet_bootstrap.cc:492] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f: Bootstrap starting.
I20260808 10:37:08.635447 13455 tablet_bootstrap.cc:654] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:08.635845 13457 tablet_bootstrap.cc:492] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: Bootstrap starting.
I20260808 10:37:08.636716 13455 log.cc:824] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:08.638372 13456 tablet_bootstrap.cc:654] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:08.640829 13457 tablet_bootstrap.cc:654] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:08.642452 13456 log.cc:824] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:08.643536 13455 tablet_bootstrap.cc:492] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: No bootstrap required, opened a new log
I20260808 10:37:08.643846 13457 log.cc:824] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:08.643903 13455 ts_tablet_manager.cc:1397] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260808 10:37:08.646695 13457 tablet_bootstrap.cc:492] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: No bootstrap required, opened a new log
I20260808 10:37:08.646960 13457 ts_tablet_manager.cc:1397] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260808 10:37:08.647053 13456 tablet_bootstrap.cc:492] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f: No bootstrap required, opened a new log
I20260808 10:37:08.647264 13456 ts_tablet_manager.cc:1397] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.002s
I20260808 10:37:08.648485 13455 raft_consensus.cc:348] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 } } peers { permanent_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44825 } } peers { permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 } }
I20260808 10:37:08.648772 13455 raft_consensus.cc:374] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:08.648815 13455 raft_consensus.cc:729] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21406271d4724f72a848035e9efbdf93, State: Initialized, Role: FOLLOWER
I20260808 10:37:08.649458 13455 consensus_queue.cc:260] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 } } peers { permanent_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44825 } } peers { permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 } }
I20260808 10:37:08.650812 13455 ts_tablet_manager.cc:1428] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:37:08.651160 13270 heartbeater.cc:503] Master 127.30.146.190:36561 was elected leader, sending a full tablet report...
I20260808 10:37:08.652617 13456 raft_consensus.cc:348] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 } } peers { permanent_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44825 } } peers { permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 } }
I20260808 10:37:08.653046 13456 raft_consensus.cc:374] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:08.653105 13456 raft_consensus.cc:729] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e8fe54c1bc4a96b331e21d0c0c735f, State: Initialized, Role: FOLLOWER
I20260808 10:37:08.653646 13456 consensus_queue.cc:260] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 } } peers { permanent_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44825 } } peers { permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 } }
I20260808 10:37:08.654536 13457 raft_consensus.cc:348] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 } } peers { permanent_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44825 } } peers { permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 } }
I20260808 10:37:08.654914 13457 raft_consensus.cc:374] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:08.655023 13456 ts_tablet_manager.cc:1428] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20260808 10:37:08.655056 13457 raft_consensus.cc:729] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9a0a6494b84c7d9c22bd31b5d4a79a, State: Initialized, Role: FOLLOWER
I20260808 10:37:08.655711 13124 heartbeater.cc:503] Master 127.30.146.190:36561 was elected leader, sending a full tablet report...
I20260808 10:37:08.655658 13457 consensus_queue.cc:260] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 } } peers { permanent_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44825 } } peers { permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 } }
I20260808 10:37:08.661999 13457 ts_tablet_manager.cc:1428] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: Time spent starting tablet: real 0.015s user 0.000s sys 0.009s
I20260808 10:37:08.662400 13419 heartbeater.cc:503] Master 127.30.146.190:36561 was elected leader, sending a full tablet report...
W20260808 10:37:08.701489 13125 tablet.cc:2369] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:37:08.758035 13271 tablet.cc:2369] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:37:08.770282 13421 tablet.cc:2369] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 10:37:08 dist-test-slave-9gcw krb5kdc[11802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185428, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for HTTP/127.30.146.148@KRBTEST.COM
I20260808 10:37:08.961997 13463 raft_consensus.cc:482] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:37:08.962338 13463 raft_consensus.cc:504] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 } } peers { permanent_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44825 } } peers { permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 } }
I20260808 10:37:08.963795 13463 leader_election.cc:288] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21406271d4724f72a848035e9efbdf93 (127.30.146.130:39569), 1c9a0a6494b84c7d9c22bd31b5d4a79a (127.30.146.131:46581)
I20260808 10:37:08.971342 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.964701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:39573 (local address 127.30.146.130:39569)
0808 10:37:08.964828 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:08.964832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:08.966030 (+ 1198us) server_negotiation.cc:407] Connection header received
0808 10:37:08.966301 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:08.966306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:08.966363 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:08.966421 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:08.966883 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.967786 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.969851 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.970451 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.970565 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.970632 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:08.971039 (+ 407us) server_negotiation.cc:299] Negotiation successful
0808 10:37:08.971183 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:37:08.972585 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.964090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:39569 (local address 127.30.146.129:39573)
0808 10:37:08.965879 (+ 1789us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:08.965898 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:08.966140 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:08.966590 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:08.966593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:08.966613 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:08.966733 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:08.966740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.967958 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.967962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:08.969699 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:08.969710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.970752 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.970756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:08.970862 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.970891 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 10:37:08.972059 (+ 1168us) client_negotiation.cc:240] Negotiation successful
0808 10:37:08.972170 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1702,"thread_start_us":218,"threads_started":1}
I20260808 10:37:08.974107 13220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6ed6e537594fceaa49909b9e7dcdc3" candidate_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21406271d4724f72a848035e9efbdf93" is_pre_election: true
I20260808 10:37:08.974491 13220 raft_consensus.cc:2454] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e8fe54c1bc4a96b331e21d0c0c735f in term 0.
I20260808 10:37:08.975169 13000 leader_election.cc:302] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21406271d4724f72a848035e9efbdf93, 91e8fe54c1bc4a96b331e21d0c0c735f; no voters:
I20260808 10:37:08.975538 13463 raft_consensus.cc:2781] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:37:08.975653 13463 raft_consensus.cc:482] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:37:08.975709 13463 raft_consensus.cc:3037] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:08.977730 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.965561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:51931 (local address 127.30.146.131:46581)
0808 10:37:08.965703 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:08.965708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:08.969576 (+ 3868us) server_negotiation.cc:407] Connection header received
0808 10:37:08.969796 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:08.969800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:08.969856 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:08.969951 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:08.972221 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.973065 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.976169 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.976773 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.976871 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.976922 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:08.977513 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 10:37:08.977600 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:37:08.977912 13463 raft_consensus.cc:504] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 } } peers { permanent_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44825 } } peers { permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 } }
I20260808 10:37:08.978219 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:08.964934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:46581 (local address 127.30.146.129:51931)
0808 10:37:08.969416 (+ 4482us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:08.969435 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:08.969659 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:08.971895 (+ 2236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:08.971898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:08.971919 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:08.972047 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:08.972054 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.973642 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.973645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:08.975999 (+ 2354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:08.976011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:08.977198 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:08.977204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:08.977360 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:08.977387 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:37:08.977921 (+ 534us) client_negotiation.cc:240] Negotiation successful
0808 10:37:08.978002 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4395,"thread_start_us":219,"threads_started":1}
I20260808 10:37:08.978384 13463 leader_election.cc:288] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [CANDIDATE]: Term 1 election: Requested vote from peers 21406271d4724f72a848035e9efbdf93 (127.30.146.130:39569), 1c9a0a6494b84c7d9c22bd31b5d4a79a (127.30.146.131:46581)
I20260808 10:37:08.979293 13352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6ed6e537594fceaa49909b9e7dcdc3" candidate_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a"
I20260808 10:37:08.979372 13371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6ed6e537594fceaa49909b9e7dcdc3" candidate_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" is_pre_election: true
I20260808 10:37:08.979553 13352 raft_consensus.cc:3037] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:08.980502 13220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a6ed6e537594fceaa49909b9e7dcdc3" candidate_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21406271d4724f72a848035e9efbdf93"
I20260808 10:37:08.980638 13220 raft_consensus.cc:3037] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:08.981927 13352 raft_consensus.cc:2454] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e8fe54c1bc4a96b331e21d0c0c735f in term 1.
I20260808 10:37:08.982739 13220 raft_consensus.cc:2454] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e8fe54c1bc4a96b331e21d0c0c735f in term 1.
I20260808 10:37:08.983083 13000 leader_election.cc:302] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21406271d4724f72a848035e9efbdf93, 91e8fe54c1bc4a96b331e21d0c0c735f; no voters:
I20260808 10:37:08.983368 13463 raft_consensus.cc:2781] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:37:08.983644 13463 raft_consensus.cc:686] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 1 LEADER]: Becoming Leader. State: Replica: 91e8fe54c1bc4a96b331e21d0c0c735f, State: Running, Role: LEADER
I20260808 10:37:08.984040 13463 consensus_queue.cc:237] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 } } peers { permanent_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44825 } } peers { permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 } }
I20260808 10:37:08.988121 12905 catalog_manager.cc:5495] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e8fe54c1bc4a96b331e21d0c0c735f (127.30.146.129). New cstate: current_term: 1 leader_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e8fe54c1bc4a96b331e21d0c0c735f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 44825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 } health_report { overall_health: UNKNOWN } } }
I20260808 10:37:09.060961 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:09.056182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.30.146.129:44825)
0808 10:37:09.056316 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:09.056322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:09.056367 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:37:09.056601 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:09.056607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:09.056732 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:09.056861 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:37:09.057346 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:09.057969 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:09.059257 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:09.059528 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:09.059657 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:09.060031 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:37:09.060493 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:37:09.060642 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:09.060724 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 10:37:09.060790 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:37:09.069097 13352 raft_consensus.cc:1261] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a [term 1 FOLLOWER]: Refusing update from remote peer 91e8fe54c1bc4a96b331e21d0c0c735f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:09.069128 13220 raft_consensus.cc:1261] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93 [term 1 FOLLOWER]: Refusing update from remote peer 91e8fe54c1bc4a96b331e21d0c0c735f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:09.069878 13463 consensus_queue.cc:1035] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c9a0a6494b84c7d9c22bd31b5d4a79a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 46581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:09.070246 13469 consensus_queue.cc:1035] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [LEADER]: Connected to new peer: Peer: permanent_uuid: "21406271d4724f72a848035e9efbdf93" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:09.079753 13481 mvcc.cc:205] Tried to move back new op lower bound from 7316215517459038208 to 7316215517123129344. Current Snapshot: MvccSnapshot[applied={T|T < 7316215517459038208}]
I20260808 10:37:09.091001 12903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:37:09.091214 12903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:37:09.094111 12903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f25ca01b923648679b613affeebb967b: Sending DeleteTablet for 3 replicas of tablet 0a6ed6e537594fceaa49909b9e7dcdc3
I20260808 10:37:09.095058 13046 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a6ed6e537594fceaa49909b9e7dcdc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:09 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:59162
I20260808 10:37:09.095129 13199 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a6ed6e537594fceaa49909b9e7dcdc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:09 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:41156
I20260808 10:37:09.095429 13346 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a6ed6e537594fceaa49909b9e7dcdc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:09 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:59342
I20260808 10:37:09.095783 13489 tablet_replica.cc:330] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f: stopping tablet replica
I20260808 10:37:09.096190 13489 raft_consensus.cc:2229] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:37:09.096175 13491 tablet_replica.cc:330] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: stopping tablet replica
I20260808 10:37:09.096666 13489 raft_consensus.cc:2258] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 91e8fe54c1bc4a96b331e21d0c0c735f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:09.096697 13491 raft_consensus.cc:2229] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:09.096956 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 12948
I20260808 10:37:09.097203 13490 tablet_replica.cc:330] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: stopping tablet replica
I20260808 10:37:09.097709 13491 raft_consensus.cc:2258] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:09.097746 13490 raft_consensus.cc:2229] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:09.098294 13490 raft_consensus.cc:2258] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:09.099640 13491 ts_tablet_manager.cc:1905] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:09.100543 13490 ts_tablet_manager.cc:1905] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:09.103168 13491 ts_tablet_manager.cc:1918] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:09.103318 13491 log.cc:1196] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/0a6ed6e537594fceaa49909b9e7dcdc3
I20260808 10:37:09.103679 13491 ts_tablet_manager.cc:1939] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 1c9a0a6494b84c7d9c22bd31b5d4a79a: Deleting consensus metadata
I20260808 10:37:09.104386 13490 ts_tablet_manager.cc:1918] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:09.104504 13490 log.cc:1196] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/0a6ed6e537594fceaa49909b9e7dcdc3
I20260808 10:37:09.104934 13490 ts_tablet_manager.cc:1939] T 0a6ed6e537594fceaa49909b9e7dcdc3 P 21406271d4724f72a848035e9efbdf93: Deleting consensus metadata
I20260808 10:37:09.104933 12891 catalog_manager.cc:4841] TS 1c9a0a6494b84c7d9c22bd31b5d4a79a (127.30.146.131:46581): tablet 0a6ed6e537594fceaa49909b9e7dcdc3 (table test-table [id=7a2edca3640b4b1dba7e70eed0c89410]) successfully deleted
I20260808 10:37:09.106272 12890 catalog_manager.cc:4841] TS 21406271d4724f72a848035e9efbdf93 (127.30.146.130:39569): tablet 0a6ed6e537594fceaa49909b9e7dcdc3 (table test-table [id=7a2edca3640b4b1dba7e70eed0c89410]) successfully deleted
W20260808 10:37:09.108111 12889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:44825 (error 108)
I20260808 10:37:09.108333 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 13128
I20260808 10:37:09.110646 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:09.110274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:44825 (local address 127.0.0.1:59170)
0808 10:37:09.110420 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:09.110517 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:37:09.110898 12889 catalog_manager.cc:4584] TS 91e8fe54c1bc4a96b331e21d0c0c735f (127.30.146.129:44825): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a6ed6e537594fceaa49909b9e7dcdc3: Network error: Client connection negotiation failed: client connection to 127.30.146.129:44825: connect: Connection refused (error 111)
I20260808 10:37:09.116560 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 13273
I20260808 10:37:09.123692 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 12869
I20260808 10:37:09.134362 31306 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 10:37:09.479611 31306 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 10:37:09.479741 31306 mini_ranger.cc:66] Stopping Ranger...
I20260808 10:37:09.835513 31306 mini_ranger.cc:68] Stopped Ranger
2026-08-08 10:37:09.835 UTC [11837] LOG: received smart shutdown request
2026-08-08 10:37:09.840 UTC [11837] LOG: background worker "logical replication launcher" (PID 11848) exited with exit code 1
2026-08-08 10:37:09.840 UTC [11842] LOG: shutting down
2026-08-08 10:37:10.027 UTC [11837] LOG: database system is shut down
2026-08-08T10:37:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51820 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:37:10 dist-test-slave-9gcw krb5kdc[13502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:37:10 dist-test-slave-9gcw krb5kdc[13502](info): set up 2 sockets
Aug 08 10:37:10 dist-test-slave-9gcw krb5kdc[13502](info): commencing operation
krb5kdc: starting...
W20260808 10:37:11.305018 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:37:11 dist-test-slave-9gcw krb5kdc[13502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:37:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:37:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:37:11.449569 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:46085
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:46085,127.30.146.189:32991,127.30.146.188:38145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:11.563716 13518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:11.564057 13518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:11.564142 13518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:11.568207 13518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:11.568305 13518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:11.568346 13518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:11.568413 13518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:11.568450 13518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:11.573743 13518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:46085,127.30.146.189:32991,127.30.146.188:38145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:46085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:11.574930 13518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:11.575996 13518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:11.584643 13527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:11.584661 13524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:11.584782 13518 server_base.cc:1034] running on GCE node
W20260808 10:37:11.584661 13525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:11.585561 13518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:11.586762 13518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:11.587960 13518 hybrid_clock.cc:648] HybridClock initialized: now 1786185431587940 us; error 31 us; skew 500 ppm
Aug 08 10:37:11 dist-test-slave-9gcw krb5kdc[13502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:11.590922 13518 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:37:11.591874 13518 webserver.cc:460] Webserver started at http://127.30.146.190:44581/ using document root <none> and password file <none>
I20260808 10:37:11.592478 13518 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:11.592572 13518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:11.592815 13518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:11.594946 13518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "301dd89df0544974bc8e14e066cf1407"
format_stamp: "Formatted at 2026-08-08 10:37:11 on dist-test-slave-9gcw"
server_key: "46e4b9de72b8b1063c9edcfb08722d10"
server_key_iv: "0e6b9c22f1e4a0761eb87e96efdbc303"
server_key_version: "encryptionkey@0"
I20260808 10:37:11.595546 13518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "301dd89df0544974bc8e14e066cf1407"
format_stamp: "Formatted at 2026-08-08 10:37:11 on dist-test-slave-9gcw"
server_key: "46e4b9de72b8b1063c9edcfb08722d10"
server_key_iv: "0e6b9c22f1e4a0761eb87e96efdbc303"
server_key_version: "encryptionkey@0"
I20260808 10:37:11.599390 13518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:37:11.602069 13534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:11.603523 13518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:11.603672 13518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "301dd89df0544974bc8e14e066cf1407"
format_stamp: "Formatted at 2026-08-08 10:37:11 on dist-test-slave-9gcw"
server_key: "46e4b9de72b8b1063c9edcfb08722d10"
server_key_iv: "0e6b9c22f1e4a0761eb87e96efdbc303"
server_key_version: "encryptionkey@0"
I20260808 10:37:11.603796 13518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:11.617556 13518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:11.621017 13518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:11.621265 13518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:11.628950 13518 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:46085
I20260808 10:37:11.629421 13591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:46085 every 8 connection(s)
I20260808 10:37:11.630048 13518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:37:11.633019 13592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:11.637238 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 13518
I20260808 10:37:11.637436 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:37:11.637725 31306 external_mini_cluster.cc:1428] Setting key 6cce93f458929b2c16b4f6d12258073a
I20260808 10:37:11.637545 13592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.146.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.189@KRBTEST.COM" created.
Aug 08 10:37:11 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:11.657196 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.640203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46085 (local address 127.0.0.1:54934)
0808 10:37:11.640838 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.640903 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:11.641919 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:11.642806 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:11.642811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:11.643267 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:11.643543 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:11.643558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.645455 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.645458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:11.646193 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:11.646201 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.647156 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.647159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:11.647321 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:11.648079 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:11.648096 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:11.651821 (+ 3725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:11.653968 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:11.653978 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:11.653989 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:11.654229 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:11.654582 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:11.654584 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:11.654586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:11.654704 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:11.655343 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:11.655349 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:11.655525 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 10:37:11.655703 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 10:37:11.656084 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":153,"spinlock_wait_cycles":8320,"thread_start_us":148,"threads_started":1}
I20260808 10:37:11.657196 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.640203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54934 (local address 127.30.146.190:46085)
0808 10:37:11.640806 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:11.640825 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:11.641047 (+ 222us) server_negotiation.cc:407] Connection header received
0808 10:37:11.642283 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:11.642291 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:11.642547 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:11.642809 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:11.643725 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.645312 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.646324 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.646978 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.647183 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:11.652003 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:11.652009 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:11.652022 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:11.652044 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:11.653832 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:11.654354 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:11.654360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:11.654362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:11.654476 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:11.654805 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:11.654809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:11.654810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:11.655211 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:11.655374 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:11.655764 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 10:37:11.656031 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":473,"spinlock_wait_cycles":22656,"thread_start_us":181,"threads_started":1}
I20260808 10:37:11.659703 13592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:37:11.660576 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.660221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:32991 (local address 127.0.0.1:51788)
0808 10:37:11.660398 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.660474 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:32991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260808 10:37:11.660821 13539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.146.189:32991: connect: Connection refused (error 111)
I20260808 10:37:11.661845 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.661623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:32991 (local address 127.0.0.1:51800)
0808 10:37:11.661725 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.661767 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:32991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:37:11.662000 13592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.189:32991: Network error: Client connection negotiation failed: client connection to 127.30.146.189:32991: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab.
Entry for principal kudu/127.30.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab.
I20260808 10:37:11.669023 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.189:32991
--webserver_interface=127.30.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab
--principal=kudu/127.30.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:46085,127.30.146.189:32991,127.30.146.188:38145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:37:11.711264 13592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } attempt: 1
I20260808 10:37:11.712258 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.711925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:32991 (local address 127.0.0.1:51804)
0808 10:37:11.712102 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.712178 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:32991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 10:37:11.713104 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.712866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:32991 (local address 127.0.0.1:51814)
0808 10:37:11.713003 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.713050 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:32991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 10:37:11.713357 13592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.189:32991: Network error: Client connection negotiation failed: client connection to 127.30.146.189:32991: connect: Connection refused (error 111)
I20260808 10:37:11.781553 13592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } attempt: 2
I20260808 10:37:11.782521 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.782205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:32991 (local address 127.0.0.1:51822)
0808 10:37:11.782371 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.782434 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:32991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 10:37:11.783383 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.783113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:32991 (local address 127.0.0.1:51832)
0808 10:37:11.783281 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.783329 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:32991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260808 10:37:11.783604 13592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.189:32991: Network error: Client connection negotiation failed: client connection to 127.30.146.189:32991: connect: Connection refused (error 111)
W20260808 10:37:11.795815 13599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:11.796149 13599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:11.796250 13599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:11.800489 13599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:11.800573 13599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:11.800597 13599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:11.800668 13599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:11.800712 13599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:11.806397 13599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:46085,127.30.146.189:32991,127.30.146.188:38145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab
--principal=kudu/127.30.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.189:32991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:11.807855 13599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:11.809058 13599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:11.817777 13604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:11.817777 13605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:11.818012 13607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:11.818296 13599 server_base.cc:1034] running on GCE node
I20260808 10:37:11.818879 13599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:11.819984 13599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:11.821200 13599 hybrid_clock.cc:648] HybridClock initialized: now 1786185431821188 us; error 33 us; skew 500 ppm
Aug 08 10:37:11 dist-test-slave-9gcw krb5kdc[13502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:11.824225 13599 init.cc:377] Logged in from keytab as kudu/127.30.146.189@KRBTEST.COM (short username kudu)
I20260808 10:37:11.825241 13599 webserver.cc:460] Webserver started at http://127.30.146.189:44017/ using document root <none> and password file <none>
I20260808 10:37:11.825819 13599 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:11.825919 13599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:11.826155 13599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:11.828332 13599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/data/instance:
uuid: "012f17cad9ee444ab33535ce78d9d8e8"
format_stamp: "Formatted at 2026-08-08 10:37:11 on dist-test-slave-9gcw"
server_key: "e9ef5459c5885359fed68c5bf4a1ccc8"
server_key_iv: "68aa7477e9338ca844b1755f8d7d585a"
server_key_version: "encryptionkey@0"
I20260808 10:37:11.828930 13599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/wal/instance:
uuid: "012f17cad9ee444ab33535ce78d9d8e8"
format_stamp: "Formatted at 2026-08-08 10:37:11 on dist-test-slave-9gcw"
server_key: "e9ef5459c5885359fed68c5bf4a1ccc8"
server_key_iv: "68aa7477e9338ca844b1755f8d7d585a"
server_key_version: "encryptionkey@0"
I20260808 10:37:11.833019 13599 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:37:11.835695 13614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:11.837239 13599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:11.837392 13599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/wal
uuid: "012f17cad9ee444ab33535ce78d9d8e8"
format_stamp: "Formatted at 2026-08-08 10:37:11 on dist-test-slave-9gcw"
server_key: "e9ef5459c5885359fed68c5bf4a1ccc8"
server_key_iv: "68aa7477e9338ca844b1755f8d7d585a"
server_key_version: "encryptionkey@0"
I20260808 10:37:11.837522 13599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:11.859406 13599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:11.863191 13599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:11.863417 13599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:11.871346 13599 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.189:32991
I20260808 10:37:11.871805 13675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.189:32991 every 8 connection(s)
I20260808 10:37:11.872429 13599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
I20260808 10:37:11.874936 13592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } attempt: 3
I20260808 10:37:11.875890 13676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:11.877471 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 13599
I20260808 10:37:11.877615 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-1/wal/instance
I20260808 10:37:11.877961 31306 external_mini_cluster.cc:1428] Setting key c3c57e73efa27973d4fca671de8be6e2
I20260808 10:37:11.881934 13676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:37:11 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.188@KRBTEST.COM; defaulting to no policy
I20260808 10:37:11.890602 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.875592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:32991 (local address 127.0.0.1:51842)
0808 10:37:11.875769 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.875791 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:11.875912 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:11.878818 (+ 2906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:11.878822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:11.879048 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:11.879207 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:11.879217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.880788 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.880792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:11.881615 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:11.881630 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.882467 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.882472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:11.882584 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:11.883200 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:11.883216 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:11.884817 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:11.888494 (+ 3677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:11.888500 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:11.888502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:11.888730 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:11.889100 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:11.889104 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:11.889106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:11.889260 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:11.890112 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:11.890114 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:11.890213 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 10:37:11.890307 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 10:37:11.890419 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 10:37:11.891940 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.876574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.30.146.189:32991)
0808 10:37:11.877065 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:11.877081 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:11.877119 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:37:11.877998 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:11.878043 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:11.878448 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:11.878820 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:11.879363 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.880614 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.881772 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.882324 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.882526 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:11.885088 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:11.885096 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:11.885108 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:11.885190 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:11.888367 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:11.888856 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:11.888867 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:11.888872 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:11.888990 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:11.889390 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:11.889394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:11.889395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:11.889984 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:11.890140 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:11.890667 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 10:37:11.891004 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":164,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.188@KRBTEST.COM" created.
Aug 08 10:37:11 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:11.894272 13592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:37:11.895345 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.895073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52550)
0808 10:37:11.895213 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.895264 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 10:37:11.896222 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.895948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52552)
0808 10:37:11.896099 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.896157 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 10:37:11.896435 13592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:38145: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
I20260808 10:37:11.897907 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.883990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.30.146.190:46085)
0808 10:37:11.884119 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:11.884127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:11.884538 (+ 411us) server_negotiation.cc:407] Connection header received
0808 10:37:11.884742 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:11.884746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:11.884796 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:11.884851 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:11.885925 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.886482 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.887948 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.889240 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.889357 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:11.893477 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:11.893485 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:11.893487 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:11.893507 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:11.895631 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:11.896256 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:11.896259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:11.896264 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:11.896355 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:11.896805 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:11.896808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:11.896927 (+ 119us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:11.897114 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:11.897254 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:11.897715 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 10:37:11.897815 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:37:11.898157 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.883987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46085 (local address 127.0.0.1:54948)
0808 10:37:11.884409 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.884466 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:11.884629 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:11.885036 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:11.885040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:11.885604 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:11.885779 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:11.885791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.886617 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.886622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:11.887796 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:11.887810 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.889373 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.889379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:11.889520 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:11.890061 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:11.890082 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:11.893337 (+ 3255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:11.895772 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:11.895783 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:11.895790 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:11.896100 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:11.896554 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:11.896557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:11.896559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:11.896659 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:11.897265 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:11.897275 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:11.897437 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 10:37:11.897611 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 10:37:11.897851 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":121,"threads_started":1}
I20260808 10:37:11.899397 13676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:37:11 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
I20260808 10:37:11.909946 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.899925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:32991 (local address 127.0.0.1:51848)
0808 10:37:11.900057 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.900070 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:11.900148 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:11.900394 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:11.900396 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:11.900565 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:11.900653 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:11.900659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.901500 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.901503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:11.902008 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:11.902016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.902451 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.902455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:11.902539 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:11.903007 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:11.903020 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:11.906093 (+ 3073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:11.908031 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:11.908033 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:11.908035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:11.908329 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:11.908614 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:11.908616 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:11.908617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:11.908712 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:11.909373 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:11.909376 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:11.909467 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 10:37:11.909606 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 10:37:11.909811 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 10:37:11.910082 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.900003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.30.146.189:32991)
0808 10:37:11.900107 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:11.900111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:11.900123 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:37:11.900261 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:11.900264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:11.900307 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:11.900383 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:11.900794 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.901390 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.902119 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:11.902344 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:11.902452 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:11.906247 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:11.906250 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:11.906251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:11.906273 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:11.907921 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:11.908440 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:11.908442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:11.908444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:11.908518 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:11.908854 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:11.908857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:11.908858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:11.909265 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:11.909368 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:11.909797 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 10:37:11.909909 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:37:11.911274 13676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:37:11.912210 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.911848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52562)
0808 10:37:11.911973 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.912133 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab.
Entry for principal kudu/127.30.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab.
W20260808 10:37:11.913051 13618 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
I20260808 10:37:11.913569 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.188:38145
--webserver_interface=127.30.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab
--principal=kudu/127.30.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:46085,127.30.146.189:32991,127.30.146.188:38145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:37:11.914160 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.913930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52578)
0808 10:37:11.914056 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.914105 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 10:37:11.914414 13676 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:38145: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
I20260808 10:37:11.927623 13592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } attempt: 1
I20260808 10:37:11.928558 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.928216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52594)
0808 10:37:11.928390 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.928463 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 10:37:11.929374 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.929086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52610)
0808 10:37:11.929251 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.929305 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:37:11.929551 13592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:38145: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
I20260808 10:37:11.963775 13676 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } attempt: 1
I20260808 10:37:11.964854 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.964419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52622)
0808 10:37:11.964596 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.964763 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 10:37:11.965770 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:11.965518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52636)
0808 10:37:11.965649 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:11.965706 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 10:37:11.965971 13676 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:38145: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
I20260808 10:37:12.004738 13592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } attempt: 2
I20260808 10:37:12.005730 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.005357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52652)
0808 10:37:12.005552 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:12.005621 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 10:37:12.006840 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.006469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52656)
0808 10:37:12.006686 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:12.006746 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 10:37:12.007087 13592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:38145: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
I20260808 10:37:12.034185 13676 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } attempt: 2
I20260808 10:37:12.035190 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.034819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52658)
0808 10:37:12.035012 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:12.035080 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260808 10:37:12.036026 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.035786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52674)
0808 10:37:12.035907 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:12.035964 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:37:12.036181 13676 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:38145: Network error: Client connection negotiation failed: client connection to 127.30.146.188:38145: connect: Connection refused (error 111)
W20260808 10:37:12.036589 13685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:12.036877 13685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:12.036969 13685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:12.041332 13685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:12.041399 13685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:12.041419 13685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:12.041436 13685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:12.041452 13685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:12.047075 13685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:46085,127.30.146.189:32991,127.30.146.188:38145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab
--principal=kudu/127.30.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.188:38145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:12.048272 13685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:12.049438 13685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:12.059063 13690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:12.059152 13691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:12.059340 13693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:12.059609 13685 server_base.cc:1034] running on GCE node
I20260808 10:37:12.060201 13685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:12.061549 13685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:12.062722 13685 hybrid_clock.cc:648] HybridClock initialized: now 1786185432062696 us; error 56 us; skew 500 ppm
Aug 08 10:37:12 dist-test-slave-9gcw krb5kdc[13502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185432, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:12.066730 13685 init.cc:377] Logged in from keytab as kudu/127.30.146.188@KRBTEST.COM (short username kudu)
I20260808 10:37:12.067776 13685 webserver.cc:460] Webserver started at http://127.30.146.188:41449/ using document root <none> and password file <none>
I20260808 10:37:12.068360 13685 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:12.068454 13685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:12.068679 13685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:12.070835 13685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/data/instance:
uuid: "3156d0fb3a3342c88b0184145b1779f8"
format_stamp: "Formatted at 2026-08-08 10:37:12 on dist-test-slave-9gcw"
server_key: "22333e8b8be9a823f4ab06f8c4564e24"
server_key_iv: "c588c7546f76272522bdec9a68309c1f"
server_key_version: "encryptionkey@0"
I20260808 10:37:12.071431 13685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/wal/instance:
uuid: "3156d0fb3a3342c88b0184145b1779f8"
format_stamp: "Formatted at 2026-08-08 10:37:12 on dist-test-slave-9gcw"
server_key: "22333e8b8be9a823f4ab06f8c4564e24"
server_key_iv: "c588c7546f76272522bdec9a68309c1f"
server_key_version: "encryptionkey@0"
I20260808 10:37:12.075822 13685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 10:37:12.078445 13700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:12.079804 13685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:12.079910 13685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/wal
uuid: "3156d0fb3a3342c88b0184145b1779f8"
format_stamp: "Formatted at 2026-08-08 10:37:12 on dist-test-slave-9gcw"
server_key: "22333e8b8be9a823f4ab06f8c4564e24"
server_key_iv: "c588c7546f76272522bdec9a68309c1f"
server_key_version: "encryptionkey@0"
I20260808 10:37:12.080001 13685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:12.093535 13685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:12.096716 13685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:12.096884 13685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:12.104588 13685 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.188:38145
I20260808 10:37:12.104921 13757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.188:38145 every 8 connection(s)
I20260808 10:37:12.105692 13685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
I20260808 10:37:12.106261 13592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } attempt: 3
I20260808 10:37:12.109097 13758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:12.111958 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 13685
I20260808 10:37:12.112107 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185256684052-31306-0/minicluster-data/master-2/wal/instance
I20260808 10:37:12.112447 31306 external_mini_cluster.cc:1428] Setting key 081914a1a1c38209de812cd2ee7c640e
I20260808 10:37:12.114106 13758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:37:12 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
Aug 08 10:37:12 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
I20260808 10:37:12.121434 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.106790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52676)
0808 10:37:12.106930 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:12.106948 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:12.107052 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:12.109561 (+ 2509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:12.109566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:12.109793 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:12.109924 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:12.109932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.111284 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.111291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:12.112207 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:12.112221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.113129 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.113133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:12.113254 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.113863 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:12.113878 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:12.115615 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:12.119401 (+ 3786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:12.119407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:12.119411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:12.119709 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:12.120122 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:12.120126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:12.120128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:12.120226 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:12.120859 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:12.120862 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:12.120970 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 10:37:12.121090 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 10:37:12.121261 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 10:37:12.122637 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.107533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.30.146.188:38145)
0808 10:37:12.108024 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.108046 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.108098 (+ 52us) server_negotiation.cc:407] Connection header received
0808 10:37:12.108829 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.108855 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.109273 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.109553 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.110096 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.111143 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.112349 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.113016 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.113197 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.115812 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.115818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.115836 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.115867 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.119276 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.119845 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.119854 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.119863 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.120028 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.120323 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.120328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.120330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.120750 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.120886 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.121273 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.121570 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":150,"threads_started":1}
Aug 08 10:37:12 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185432, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:12.125888 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.113261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.30.146.188:38145)
0808 10:37:12.113681 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.113697 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.114519 (+ 822us) server_negotiation.cc:407] Connection header received
0808 10:37:12.114662 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.114665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.114716 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.114809 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.115644 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.116459 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.117708 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.118083 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.118223 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.121405 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.121410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.121414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.121445 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.123500 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.124165 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.124168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.124170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.124301 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.124613 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.124616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.124618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.124857 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.125020 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.125450 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.125748 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":164,"threads_started":1}
I20260808 10:37:12.127417 13592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407: Bootstrap starting.
I20260808 10:37:12.127377 13676 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } attempt: 3
I20260808 10:37:12.130074 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.117091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.30.146.190:46085)
0808 10:37:12.117266 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.117270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.117292 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:37:12.117321 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.117323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.117357 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.117454 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.118299 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.119332 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.120812 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.121251 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.121341 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.125698 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.125702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.125704 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.125730 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.127560 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.128191 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.128195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.128197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.128295 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.128653 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.128660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.128665 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.128988 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.129069 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.129781 (+ 712us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.129908 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":78}
I20260808 10:37:12.130383 13592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:12.130657 13764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.116261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:46085 (local address 127.0.0.1:54952)
0808 10:37:12.116689 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:12.116743 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:12.117138 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:12.117489 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:12.117495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:12.118011 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:12.118151 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:12.118163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.119514 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.119517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:12.120645 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:12.120660 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.121512 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.121515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:12.121663 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.122605 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:12.122630 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:12.125491 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:12.127704 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:12.127718 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:12.127727 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:12.128043 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:12.128421 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:12.128426 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:12.128431 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:12.128529 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:12.129275 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:12.129281 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:12.129506 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 10:37:12.130252 (+ 746us) client_negotiation.cc:240] Negotiation successful
0808 10:37:12.130469 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":45,"thread_start_us":105,"threads_started":1}
I20260808 10:37:12.131315 13592 log.cc:824] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:12.132186 13758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:37:12.134013 13592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407: No bootstrap required, opened a new log
Aug 08 10:37:12 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
Aug 08 10:37:12 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:12.138710 13592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } }
I20260808 10:37:12.138944 13592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:12.139001 13592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301dd89df0544974bc8e14e066cf1407, State: Initialized, Role: FOLLOWER
Aug 08 10:37:12 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185432, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
I20260808 10:37:12.139506 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.127284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54958 (local address 127.30.146.190:46085)
0808 10:37:12.127847 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.127860 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.127873 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:37:12.127905 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.127908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.127947 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.128761 (+ 814us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.128928 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.129773 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.130864 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.131211 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.131315 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.135234 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.135238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.135240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.135264 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.137076 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.137616 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.137620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.137621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.137732 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.138046 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.138049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.138051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.138258 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.139112 (+ 854us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.139189 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.139323 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":496,"thread_start_us":111,"threads_started":1}
I20260808 10:37:12.139525 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.127839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52700)
0808 10:37:12.128124 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:12.128141 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:12.128218 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:12.128767 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:12.128770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:12.128959 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:12.129085 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:12.129096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.130417 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.130423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:12.131249 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:12.131259 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.132215 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.132219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:12.132335 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.132821 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:12.132843 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:12.134463 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:12.137294 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:12.137297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:12.137300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:12.137579 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:12.138285 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:12.138292 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:12.138296 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:12.138396 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:12.138942 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:12.138946 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:12.139062 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 10:37:12.139175 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 10:37:12.139315 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260808 10:37:12.140121 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.128379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52700 (local address 127.30.146.188:38145)
0808 10:37:12.128501 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.128505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.128516 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:37:12.128550 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.128556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.128623 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.129256 (+ 633us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.129456 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.130275 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.131621 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.132051 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.132588 (+ 537us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.134614 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.134618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.134621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.134647 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.137137 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.137726 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.137732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.137735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.138149 (+ 414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.138527 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.138532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.138537 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.138779 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.139720 (+ 941us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.139807 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.139928 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":49}
I20260808 10:37:12.140275 13592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } }
I20260808 10:37:12.140784 13767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } } }
I20260808 10:37:12.140936 13767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:37:12.141901 13592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:12.144797 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.133553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.30.146.189:32991)
0808 10:37:12.133686 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.133693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.133711 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:37:12.133741 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.133743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.133778 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.133880 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.134348 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.135216 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.136758 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.137208 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.137294 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.140012 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.140016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.140019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.140044 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.142735 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.143267 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.143271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.143273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.143363 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.143711 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.143715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.143717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.144003 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.144083 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.144486 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.144620 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:37:12.144996 13676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8: Bootstrap starting.
W20260808 10:37:12.147598 13779 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:37:12.147722 13779 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:37:12.148989 13676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:12.150233 13676 log.cc:824] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:12.147265 13518 master_runner.cc:186] Error getting master registration for 127.30.146.189:32991: OK, instance_id { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" instance_seqno: 1786185431869190 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:37:12.150784 13518 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:37:12.151317 13764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.132725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:32991 (local address 127.0.0.1:51852)
0808 10:37:12.132953 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:12.132966 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:12.133035 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:12.133897 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:12.133901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:12.134074 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:12.134201 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:12.134210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.135398 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.135402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:12.136591 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:12.136615 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.137453 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.137459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:12.137588 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.138110 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:12.138127 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:12.139856 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:12.142870 (+ 3014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:12.142873 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:12.142875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:12.143143 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:12.143491 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:12.143495 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:12.143497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:12.143591 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:12.144177 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:12.144179 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:12.144283 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 10:37:12.150846 (+ 6563us) client_negotiation.cc:240] Negotiation successful
0808 10:37:12.151071 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"spinlock_wait_cycles":768}
Aug 08 10:37:12 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
I20260808 10:37:12.152180 13758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:37:12.155668 13676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8: No bootstrap required, opened a new log
I20260808 10:37:12.156821 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.141709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51864 (local address 127.30.146.189:32991)
0808 10:37:12.144969 (+ 3260us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.144973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.144988 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:37:12.145025 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.145028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.145075 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.145193 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.146175 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.146927 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.148145 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.148495 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.148598 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.152106 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.152110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.152113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.152138 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.154228 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.154763 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.154767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.154769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.154862 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.155213 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.155216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.155218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.155478 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.155661 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.156474 (+ 813us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.156605 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3198,"thread_start_us":107,"threads_started":1}
Aug 08 10:37:12 dist-test-slave-9gcw krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185432, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
I20260808 10:37:12.160605 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:37:12.162352 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.152943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.30.146.188:38145)
0808 10:37:12.153091 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.153096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.153110 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:37:12.153282 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.153286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.153338 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.153420 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.153927 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.154686 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.155810 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.156123 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.156206 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.158851 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.158854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.158857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.158881 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.160611 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.161080 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.161083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.161085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.161204 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.161479 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.161482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.161484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.161686 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.161756 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.162058 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.162190 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 10:37:12.162803 13764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.152894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:38145 (local address 127.0.0.1:52702)
0808 10:37:12.153005 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:12.153021 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:12.153120 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:12.153522 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:12.153525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:12.153707 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:12.153803 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:12.153810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.154810 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.154815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:12.155686 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:12.155697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.156350 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.156353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:12.156430 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.156906 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:12.156923 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:12.158693 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:12.160730 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:12.160733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:12.160735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:12.160976 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:12.161309 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:12.161312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:12.161315 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:12.161394 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:12.161814 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:12.161817 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:12.161900 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 10:37:12.162526 (+ 626us) client_negotiation.cc:240] Negotiation successful
0808 10:37:12.162656 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:37:12.162793 13676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } }
I20260808 10:37:12.163081 13676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:12.163158 13676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f17cad9ee444ab33535ce78d9d8e8, State: Initialized, Role: FOLLOWER
I20260808 10:37:12.163794 13676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } }
I20260808 10:37:12.164434 13786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } } }
I20260808 10:37:12.164605 13786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:37:12.165520 13676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:12.166447 13758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8: Bootstrap starting.
I20260808 10:37:12.170243 13758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8: Neither blocks nor log segments found. Creating new log.
W20260808 10:37:12.171145 13798 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:37:12.171274 13798 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:37:12.171448 13758 log.cc:824] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:12.174183 13758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8: No bootstrap required, opened a new log
I20260808 10:37:12.174495 13599 master_runner.cc:186] Error getting master registration for 127.30.146.188:38145: OK, instance_id { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" instance_seqno: 1786185432102611 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:37:12.178493 13599 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:37:12.179059 13758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } }
I20260808 10:37:12.179376 13758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:12.179445 13758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3156d0fb3a3342c88b0184145b1779f8, State: Initialized, Role: FOLLOWER
I20260808 10:37:12.180246 13758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } }
I20260808 10:37:12.182422 13758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:12.186254 13805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } } }
I20260808 10:37:12.186515 13805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:37:12.190629 13817 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 10:37:12.193696 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.180072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51866 (local address 127.30.146.189:32991)
0808 10:37:12.180225 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.180231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.181718 (+ 1487us) server_negotiation.cc:407] Connection header received
0808 10:37:12.181948 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.181952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.182006 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.182069 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.183305 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.184167 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.185699 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.186105 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.186221 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.188629 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.188638 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.188645 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.188680 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.190941 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.191587 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.191592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.191594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.191704 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.192109 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.192115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.192117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.192722 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.192841 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.193323 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.193457 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:37:12.191008 13685 master_runner.cc:263] No leader master found from master 3156d0fb3a3342c88b0184145b1779f8
I20260808 10:37:12.195811 13685 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 10:37:12.195886 13817 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:37:12.202521 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.180844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52714 (local address 127.30.146.188:38145)
0808 10:37:12.180975 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.180981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.191165 (+ 10184us) server_negotiation.cc:407] Connection header received
0808 10:37:12.191391 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.191396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.191450 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.191519 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.192658 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.193500 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.194765 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.195110 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.195207 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.197027 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.197032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.197036 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.197067 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.199911 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.200594 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.200601 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.200605 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.200712 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.201118 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.201122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.201126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.201448 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.201621 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.202113 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.202265 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:37:12.203063 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:12.179259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54970 (local address 127.30.146.190:46085)
0808 10:37:12.179414 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:12.179419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:12.181298 (+ 1879us) server_negotiation.cc:407] Connection header received
0808 10:37:12.181474 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:12.181477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:12.181529 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:12.181624 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:12.182700 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.183523 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.191336 (+ 7813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:12.191726 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:12.191798 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:12.198541 (+ 6743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:12.198548 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:12.198553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:12.198593 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:12.200664 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.201310 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.201314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.201316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.201414 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:12.201804 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:12.201808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:12.201810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:12.202105 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:12.202271 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:12.202730 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 10:37:12.202878 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:37:12.340207 13767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:37:12.340440 13767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } }
I20260808 10:37:12.341856 13767 leader_election.cc:288] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f17cad9ee444ab33535ce78d9d8e8 (127.30.146.189:32991), 3156d0fb3a3342c88b0184145b1779f8 (127.30.146.188:38145)
I20260808 10:37:12.342728 13729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "301dd89df0544974bc8e14e066cf1407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3156d0fb3a3342c88b0184145b1779f8" is_pre_election: true
I20260808 10:37:12.343050 13729 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301dd89df0544974bc8e14e066cf1407 in term 0.
I20260808 10:37:12.343704 13539 leader_election.cc:302] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301dd89df0544974bc8e14e066cf1407, 3156d0fb3a3342c88b0184145b1779f8; no voters:
I20260808 10:37:12.344498 13646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "301dd89df0544974bc8e14e066cf1407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f17cad9ee444ab33535ce78d9d8e8" is_pre_election: true
I20260808 10:37:12.344816 13646 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301dd89df0544974bc8e14e066cf1407 in term 0.
I20260808 10:37:12.345551 13767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:37:12.345647 13767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:37:12.345695 13767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:12.347105 13767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } }
I20260808 10:37:12.347440 13767 leader_election.cc:288] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [CANDIDATE]: Term 1 election: Requested vote from peers 012f17cad9ee444ab33535ce78d9d8e8 (127.30.146.189:32991), 3156d0fb3a3342c88b0184145b1779f8 (127.30.146.188:38145)
I20260808 10:37:12.348007 13729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "301dd89df0544974bc8e14e066cf1407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3156d0fb3a3342c88b0184145b1779f8"
I20260808 10:37:12.348145 13729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [term 0 FOLLOWER]: Advancing to term 1
W20260808 10:37:12.348304 13706 tablet.cc:2369] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:37:12.349458 13646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "301dd89df0544974bc8e14e066cf1407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f17cad9ee444ab33535ce78d9d8e8"
I20260808 10:37:12.349579 13646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:12.350862 13646 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301dd89df0544974bc8e14e066cf1407 in term 1.
I20260808 10:37:12.351305 13539 leader_election.cc:302] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f17cad9ee444ab33535ce78d9d8e8, 301dd89df0544974bc8e14e066cf1407; no voters:
I20260808 10:37:12.351905 13729 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301dd89df0544974bc8e14e066cf1407 in term 1.
I20260808 10:37:12.352399 13767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:37:12.352669 13767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [term 1 LEADER]: Becoming Leader. State: Replica: 301dd89df0544974bc8e14e066cf1407, State: Running, Role: LEADER
I20260808 10:37:12.353051 13767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } }
I20260808 10:37:12.355077 13767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 301dd89df0544974bc8e14e066cf1407. Latest consensus state: current_term: 1 leader_uuid: "301dd89df0544974bc8e14e066cf1407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } } }
I20260808 10:37:12.355305 13767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:12.358278 13821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:37:12.359439 13821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:37:12.365384 13729 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [term 1 FOLLOWER]: Refusing update from remote peer 301dd89df0544974bc8e14e066cf1407: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 10:37:12.365480 13620 tablet.cc:2369] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:37:12.366501 13819 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:12.370980 13805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 301dd89df0544974bc8e14e066cf1407. Latest consensus state: current_term: 1 leader_uuid: "301dd89df0544974bc8e14e066cf1407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } } }
I20260808 10:37:12.371125 13805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:37:12.372298 13646 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [term 1 FOLLOWER]: Refusing update from remote peer 301dd89df0544974bc8e14e066cf1407: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:12.372933 13819 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:37:12.373097 13541 tablet.cc:2369] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:37:12.377504 13786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 301dd89df0544974bc8e14e066cf1407. Latest consensus state: current_term: 1 leader_uuid: "301dd89df0544974bc8e14e066cf1407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } } }
I20260808 10:37:12.377655 13786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:37:12.384027 13819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "301dd89df0544974bc8e14e066cf1407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } } }
I20260808 10:37:12.384294 13819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:12.385326 13767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "301dd89df0544974bc8e14e066cf1407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } } }
I20260808 10:37:12.385449 13767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:12.385648 13805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "301dd89df0544974bc8e14e066cf1407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } } }
I20260808 10:37:12.385771 13805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:37:12.386722 13786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "301dd89df0544974bc8e14e066cf1407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301dd89df0544974bc8e14e066cf1407" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 46085 } } peers { permanent_uuid: "012f17cad9ee444ab33535ce78d9d8e8" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 32991 } } peers { permanent_uuid: "3156d0fb3a3342c88b0184145b1779f8" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 38145 } } }
I20260808 10:37:12.386863 13786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:37:12.388849 13823 mvcc.cc:205] Tried to move back new op lower bound from 7316215530961154048 to 7316215530921848832. Current Snapshot: MvccSnapshot[applied={T|T < 7316215530961154048}]
I20260808 10:37:12.389989 13825 mvcc.cc:205] Tried to move back new op lower bound from 7316215530961154048 to 7316215530921848832. Current Snapshot: MvccSnapshot[applied={T|T < 7316215530961154048}]
I20260808 10:37:12.391018 13821 catalog_manager.cc:1347] Generated new cluster ID: 27fa599e8e6c4096ace76c3e3951f5ac
I20260808 10:37:12.391077 13821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:37:12.403685 13821 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:37:12.404776 13821 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:37:12.412379 13821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 301dd89df0544974bc8e14e066cf1407: Generated new TSK 0
I20260808 10:37:12.413280 13821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:37:13.172824 13798 catalog_manager.cc:1259] Loaded cluster ID: 27fa599e8e6c4096ace76c3e3951f5ac
I20260808 10:37:13.172972 13798 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8: loading cluster ID for follower catalog manager: success
I20260808 10:37:13.174908 13798 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8: acquiring CA information for follower catalog manager: success
I20260808 10:37:13.176308 13798 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 012f17cad9ee444ab33535ce78d9d8e8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:37:13.197340 13817 catalog_manager.cc:1259] Loaded cluster ID: 27fa599e8e6c4096ace76c3e3951f5ac
I20260808 10:37:13.197428 13817 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8: loading cluster ID for follower catalog manager: success
I20260808 10:37:13.198997 13817 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8: acquiring CA information for follower catalog manager: success
I20260808 10:37:13.200098 13817 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3156d0fb3a3342c88b0184145b1779f8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:37:13.473656 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 13518
I20260808 10:37:13.482913 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 13599
I20260808 10:37:13.491770 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 13685
2026-08-08T10:37:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3331 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:37:13 dist-test-slave-9gcw krb5kdc[13839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:37:13 dist-test-slave-9gcw krb5kdc[13839](info): set up 2 sockets
Aug 08 10:37:13 dist-test-slave-9gcw krb5kdc[13839](info): commencing operation
krb5kdc: starting...
W20260808 10:37:14.658195 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.006s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:37:14 dist-test-slave-9gcw krb5kdc[13839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:37:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:37:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:37:14.800449 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:41175
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:38527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:41175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:14.919552 13854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:14.919906 13854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:14.920002 13854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:14.925722 13854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:14.925909 13854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:14.925956 13854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:14.925999 13854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:14.926049 13854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:14.934103 13854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:38527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:41175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:41175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:14.935750 13854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:14.937004 13854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:14.945896 13860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:14.945927 13862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:14.946043 13864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:14.946070 13854 server_base.cc:1034] running on GCE node
I20260808 10:37:14.946918 13854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:14.948042 13854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:14.949249 13854 hybrid_clock.cc:648] HybridClock initialized: now 1786185434949234 us; error 36 us; skew 500 ppm
Aug 08 10:37:14 dist-test-slave-9gcw krb5kdc[13839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185434, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:14.952211 13854 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:37:14.953285 13854 webserver.cc:460] Webserver started at http://127.30.146.190:46673/ using document root <none> and password file <none>
I20260808 10:37:14.953877 13854 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:14.953989 13854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:14.954242 13854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:14.956435 13854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "d317951ab4b74b879bff7e1d20159641"
format_stamp: "Formatted at 2026-08-08 10:37:14 on dist-test-slave-9gcw"
server_key: "b8d3ad1b6c4365986ef4d47190e5dc37"
server_key_iv: "b24123b5e23aee391af7a30af80eb039"
server_key_version: "encryptionkey@0"
I20260808 10:37:14.957046 13854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "d317951ab4b74b879bff7e1d20159641"
format_stamp: "Formatted at 2026-08-08 10:37:14 on dist-test-slave-9gcw"
server_key: "b8d3ad1b6c4365986ef4d47190e5dc37"
server_key_iv: "b24123b5e23aee391af7a30af80eb039"
server_key_version: "encryptionkey@0"
I20260808 10:37:14.961114 13854 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:37:14.963783 13871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:14.965131 13854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:37:14.965360 13854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "d317951ab4b74b879bff7e1d20159641"
format_stamp: "Formatted at 2026-08-08 10:37:14 on dist-test-slave-9gcw"
server_key: "b8d3ad1b6c4365986ef4d47190e5dc37"
server_key_iv: "b24123b5e23aee391af7a30af80eb039"
server_key_version: "encryptionkey@0"
I20260808 10:37:14.965492 13854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:14.986975 13854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:14.990571 13854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:14.990818 13854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:14.998171 13854 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:41175
I20260808 10:37:14.998186 13927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:41175 every 8 connection(s)
I20260808 10:37:14.999191 13854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:37:15.002172 13930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:15.002684 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 13854
I20260808 10:37:15.002828 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:37:15.003163 31306 external_mini_cluster.cc:1428] Setting key 92f9873146694fb244defe5bbacff61d
I20260808 10:37:15.009876 13930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641: Bootstrap starting.
I20260808 10:37:15.012485 13930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641: Neither blocks nor log segments found. Creating new log.
Aug 08 10:37:15 dist-test-slave-9gcw krb5kdc[13839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:15.013444 13930 log.cc:824] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:15.016002 13930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641: No bootstrap required, opened a new log
I20260808 10:37:15.019260 13933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:15.004511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58450 (local address 127.30.146.190:41175)
0808 10:37:15.004946 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:15.004966 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:15.005009 (+ 43us) server_negotiation.cc:407] Connection header received
0808 10:37:15.005870 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:15.005895 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:15.006329 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:15.006711 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:15.007467 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:15.009022 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:15.009960 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:15.010644 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:15.010805 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:15.013725 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:15.013732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:15.013744 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:15.013772 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:15.015890 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:15.016374 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:15.016382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:15.016390 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:15.016529 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:15.016814 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:15.016817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:15.016819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:15.017422 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:15.017630 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:15.017889 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 10:37:15.018285 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":129,"threads_started":1}
I20260808 10:37:15.020761 13930 raft_consensus.cc:348] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317951ab4b74b879bff7e1d20159641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41175 } }
I20260808 10:37:15.020982 13930 raft_consensus.cc:374] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:15.021049 13930 raft_consensus.cc:729] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d317951ab4b74b879bff7e1d20159641, State: Initialized, Role: FOLLOWER
I20260808 10:37:15.021575 13930 consensus_queue.cc:260] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317951ab4b74b879bff7e1d20159641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41175 } }
I20260808 10:37:15.021756 13930 raft_consensus.cc:388] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:37:15.021831 13930 raft_consensus.cc:482] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:37:15.021941 13930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:15.022404 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:37:15.023173 13930 raft_consensus.cc:504] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317951ab4b74b879bff7e1d20159641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41175 } }
I20260808 10:37:15.023514 13930 leader_election.cc:302] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d317951ab4b74b879bff7e1d20159641; no voters:
I20260808 10:37:15.023881 13930 leader_election.cc:288] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:37:15.024004 13936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:37:15.024219 13936 raft_consensus.cc:686] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [term 1 LEADER]: Becoming Leader. State: Replica: d317951ab4b74b879bff7e1d20159641, State: Running, Role: LEADER
I20260808 10:37:15.024525 13936 consensus_queue.cc:237] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317951ab4b74b879bff7e1d20159641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41175 } }
I20260808 10:37:15.025131 13930 sys_catalog.cc:564] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:15.026444 13937 sys_catalog.cc:455] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d317951ab4b74b879bff7e1d20159641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317951ab4b74b879bff7e1d20159641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41175 } } }
I20260808 10:37:15.026594 13937 sys_catalog.cc:458] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:15.026445 13938 sys_catalog.cc:455] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d317951ab4b74b879bff7e1d20159641. Latest consensus state: current_term: 1 leader_uuid: "d317951ab4b74b879bff7e1d20159641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d317951ab4b74b879bff7e1d20159641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 41175 } } }
I20260808 10:37:15.026676 13938 sys_catalog.cc:458] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641 [sys.catalog]: This master's current role is: LEADER
W20260808 10:37:15.030171 13953 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:37:15.030252 13953 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:37:15.030336 13947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:37:15.031037 13947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:37:15.036345 13947 catalog_manager.cc:1347] Generated new cluster ID: c3d0d89b7ee6490786a73e84e0e13e99
I20260808 10:37:15.036427 13947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:37:15.049464 13947 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:37:15.050881 13947 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:37:15.063661 13947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d317951ab4b74b879bff7e1d20159641: Generated new TSK 0
I20260808 10:37:15.064523 13947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:37:15.079226 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 13854
2026-08-08T10:37:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1583 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:37:15 dist-test-slave-9gcw krb5kdc[13961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:37:15 dist-test-slave-9gcw krb5kdc[13961](info): set up 2 sockets
Aug 08 10:37:15 dist-test-slave-9gcw krb5kdc[13961](info): commencing operation
krb5kdc: starting...
W20260808 10:37:16.209939 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:37:16 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:37:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:37:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:37:16.350072 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33057
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:33057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:16.465082 13975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:16.465430 13975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:16.465472 13975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:16.469508 13975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:16.469568 13975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:16.469587 13975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:16.469604 13975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:16.469619 13975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:16.475095 13975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:33057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:16.476174 13975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:16.477208 13975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:16.485903 13981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:16.485896 13983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:16.485967 13980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:16.486352 13975 server_base.cc:1034] running on GCE node
I20260808 10:37:16.486950 13975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:16.488123 13975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:16.489286 13975 hybrid_clock.cc:648] HybridClock initialized: now 1786185436489273 us; error 26 us; skew 500 ppm
Aug 08 10:37:16 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185436, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:16.492384 13975 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:37:16.493474 13975 webserver.cc:460] Webserver started at http://127.30.146.190:38113/ using document root <none> and password file <none>
I20260808 10:37:16.494115 13975 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:16.494223 13975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:16.494478 13975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:16.496610 13975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "c2b184d8c0674d81b9aac53d8717d8b0"
format_stamp: "Formatted at 2026-08-08 10:37:16 on dist-test-slave-9gcw"
server_key: "8ab563a642febd3aa3e0abee657c30d0"
server_key_iv: "08c23d8c8c87719a20752ab2d6836bff"
server_key_version: "encryptionkey@0"
I20260808 10:37:16.497287 13975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "c2b184d8c0674d81b9aac53d8717d8b0"
format_stamp: "Formatted at 2026-08-08 10:37:16 on dist-test-slave-9gcw"
server_key: "8ab563a642febd3aa3e0abee657c30d0"
server_key_iv: "08c23d8c8c87719a20752ab2d6836bff"
server_key_version: "encryptionkey@0"
I20260808 10:37:16.501282 13975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:37:16.503973 13991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:16.505463 13975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:16.505630 13975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "c2b184d8c0674d81b9aac53d8717d8b0"
format_stamp: "Formatted at 2026-08-08 10:37:16 on dist-test-slave-9gcw"
server_key: "8ab563a642febd3aa3e0abee657c30d0"
server_key_iv: "08c23d8c8c87719a20752ab2d6836bff"
server_key_version: "encryptionkey@0"
I20260808 10:37:16.505752 13975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:16.523008 13975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:16.526471 13975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:16.526710 13975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:16.534549 13975 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:33057
I20260808 10:37:16.534578 14047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:33057 every 8 connection(s)
I20260808 10:37:16.535550 13975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:37:16.538365 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 13975
I20260808 10:37:16.538466 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:37:16.538682 14048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:16.538811 31306 external_mini_cluster.cc:1428] Setting key a09f498c68d4971089ca81c44f561afa
I20260808 10:37:16.545459 14048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Bootstrap starting.
Aug 08 10:37:16 dist-test-slave-9gcw krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:16.548341 14048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:16.549482 14048 log.cc:824] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:16.552167 14048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: No bootstrap required, opened a new log
I20260808 10:37:16.554642 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:16.540303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.30.146.190:33057)
0808 10:37:16.540807 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:16.540824 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:16.540860 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:37:16.541638 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:16.541656 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:16.542029 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:16.542368 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:16.543164 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:16.544145 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:16.544990 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:16.545528 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:16.545706 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:16.548777 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:16.548784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:16.548803 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:16.548840 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:16.551111 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:16.551564 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:16.551571 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:16.551578 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:16.551758 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:16.552046 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:16.552050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:16.552052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:16.552647 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:16.552866 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:16.553129 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 10:37:16.553499 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":150,"threads_started":1}
I20260808 10:37:16.557384 14048 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:16.557641 14048 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:16.557696 14048 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b184d8c0674d81b9aac53d8717d8b0, State: Initialized, Role: FOLLOWER
I20260808 10:37:16.558156 14048 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:16.558316 14048 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:37:16.558368 14048 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:37:16.558454 14048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:16.559521 14048 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:16.559871 14048 leader_election.cc:302] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2b184d8c0674d81b9aac53d8717d8b0; no voters:
I20260808 10:37:16.560178 14048 leader_election.cc:288] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:37:16.560359 14053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:37:16.560628 14053 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 1 LEADER]: Becoming Leader. State: Replica: c2b184d8c0674d81b9aac53d8717d8b0, State: Running, Role: LEADER
I20260808 10:37:16.560994 14053 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:16.561671 14048 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:16.562737 14054 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } } }
I20260808 10:37:16.562876 14054 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:16.563400 14062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:37:16.563592 14055 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2b184d8c0674d81b9aac53d8717d8b0. Latest consensus state: current_term: 1 leader_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } } }
I20260808 10:37:16.563761 14055 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:16.567015 14062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:37:16.572830 14062 catalog_manager.cc:1347] Generated new cluster ID: a7f1ea174d8d4918a092746bdb34cdd0
I20260808 10:37:16.572908 14062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:37:16.604679 14062 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:37:16.606009 14062 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:37:16.618587 14062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Generated new TSK 0
I20260808 10:37:16.619505 14062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:37:16.660758 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33057
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:16.783943 14074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:16.784264 14074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:16.784319 14074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:16.789208 14074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:16.789296 14074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:16.789419 14074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:37:16.795523 14074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:16.796768 14074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:16.797906 14074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:16.809019 14081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:16.809019 14079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:16.809021 14083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:16.809310 14074 server_base.cc:1034] running on GCE node
I20260808 10:37:16.809917 14074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:16.810724 14074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:16.811944 14074 hybrid_clock.cc:648] HybridClock initialized: now 1786185436811906 us; error 51 us; skew 500 ppm
Aug 08 10:37:16 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185436, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:16.815110 14074 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:37:16.816121 14074 webserver.cc:460] Webserver started at http://127.30.146.129:45269/ using document root <none> and password file <none>
I20260808 10:37:16.816752 14074 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:16.816850 14074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:16.817095 14074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:16.819461 14074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "b9175007187148cd8c27f9344578de42"
format_stamp: "Formatted at 2026-08-08 10:37:16 on dist-test-slave-9gcw"
server_key: "ee5a3d3262304e6e058fcf5437ee1f76"
server_key_iv: "71a35e03f578d7f2c4c2b3d5d184ac1a"
server_key_version: "encryptionkey@0"
I20260808 10:37:16.820068 14074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "b9175007187148cd8c27f9344578de42"
format_stamp: "Formatted at 2026-08-08 10:37:16 on dist-test-slave-9gcw"
server_key: "ee5a3d3262304e6e058fcf5437ee1f76"
server_key_iv: "71a35e03f578d7f2c4c2b3d5d184ac1a"
server_key_version: "encryptionkey@0"
I20260808 10:37:16.824100 14074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:37:16.826927 14091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:16.828328 14074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:37:16.828470 14074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "b9175007187148cd8c27f9344578de42"
format_stamp: "Formatted at 2026-08-08 10:37:16 on dist-test-slave-9gcw"
server_key: "ee5a3d3262304e6e058fcf5437ee1f76"
server_key_iv: "71a35e03f578d7f2c4c2b3d5d184ac1a"
server_key_version: "encryptionkey@0"
I20260808 10:37:16.828593 14074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:16.843334 14074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:16.846873 14074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:16.847105 14074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:16.847767 14074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:16.848658 14074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:16.848738 14074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:16.848817 14074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:16.848853 14074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:16.858659 14074 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:36921
I20260808 10:37:16.858718 14216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:36921 every 8 connection(s)
I20260808 10:37:16.859684 14074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:37:16.869731 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 14074
I20260808 10:37:16.869894 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:37:16.870234 31306 external_mini_cluster.cc:1428] Setting key c4701718481a64442fa5e57e1dc4355c
Aug 08 10:37:16 dist-test-slave-9gcw krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185436, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:16.875120 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:16.861643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:43035 (local address 127.30.146.190:33057)
0808 10:37:16.861801 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:16.861807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:16.862806 (+ 999us) server_negotiation.cc:407] Connection header received
0808 10:37:16.864401 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:16.864404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:16.864468 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:16.864582 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:16.865949 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:16.866523 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:16.867761 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:16.868008 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:16.868123 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:16.871292 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:16.871294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:16.871296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:16.871322 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:16.872885 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:16.873528 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:16.873531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:16.873533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:16.873613 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:16.873981 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:16.873985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:16.873987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:16.874182 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:16.874292 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:16.874825 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 10:37:16.874954 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:37:16.876089 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:16.862100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:43035)
0808 10:37:16.862581 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:16.862678 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:16.864167 (+ 1489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:16.864779 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:16.864787 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:16.865258 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:16.865770 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:16.865784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:16.866671 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:16.866675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:16.867605 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:16.867615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:16.868184 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:16.868188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:16.868367 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:16.869072 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:16.869089 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:16.871101 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:16.873018 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:16.873024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:16.873041 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:16.873405 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:16.873713 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:16.873716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:16.873719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:16.873874 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:16.874293 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:16.874301 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:16.874574 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 10:37:16.874770 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 10:37:16.875119 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":111,"threads_started":1}
I20260808 10:37:16.877509 14217 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33057
I20260808 10:37:16.877763 14217 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:16.878300 14217 heartbeater.cc:511] Master 127.30.146.190:33057 requested a full tablet report, sending...
I20260808 10:37:16.879871 14008 ts_manager.cc:194] Registered new tserver with Master: b9175007187148cd8c27f9344578de42 (127.30.146.129:36921)
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:37:16.881080 14008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:43035
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:37:16.904596 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33057
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:17.024386 14222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:17.024693 14222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:17.024767 14222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:17.028822 14222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:17.028923 14222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:17.029021 14222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:37:17.034723 14222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:17.035902 14222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:17.036973 14222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:17.047780 14229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:17.047839 14231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:17.047925 14222 server_base.cc:1034] running on GCE node
W20260808 10:37:17.047863 14228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:17.048478 14222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:17.049372 14222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:17.050545 14222 hybrid_clock.cc:648] HybridClock initialized: now 1786185437050524 us; error 34 us; skew 500 ppm
Aug 08 10:37:17 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185437, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:17.053814 14222 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:37:17.054864 14222 webserver.cc:460] Webserver started at http://127.30.146.130:35787/ using document root <none> and password file <none>
I20260808 10:37:17.055514 14222 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:17.055620 14222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:17.055889 14222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:17.058319 14222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "119a63ff2f7d497ba776b91b47e67def"
format_stamp: "Formatted at 2026-08-08 10:37:17 on dist-test-slave-9gcw"
server_key: "aaabfe5acdccc59e34f81d66cb557da3"
server_key_iv: "4a4adf50e02a8aec1205e4da3e0bd3e3"
server_key_version: "encryptionkey@0"
I20260808 10:37:17.058974 14222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "119a63ff2f7d497ba776b91b47e67def"
format_stamp: "Formatted at 2026-08-08 10:37:17 on dist-test-slave-9gcw"
server_key: "aaabfe5acdccc59e34f81d66cb557da3"
server_key_iv: "4a4adf50e02a8aec1205e4da3e0bd3e3"
server_key_version: "encryptionkey@0"
I20260808 10:37:17.063767 14222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:37:17.066793 14240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:17.068230 14222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:37:17.068390 14222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "119a63ff2f7d497ba776b91b47e67def"
format_stamp: "Formatted at 2026-08-08 10:37:17 on dist-test-slave-9gcw"
server_key: "aaabfe5acdccc59e34f81d66cb557da3"
server_key_iv: "4a4adf50e02a8aec1205e4da3e0bd3e3"
server_key_version: "encryptionkey@0"
I20260808 10:37:17.068526 14222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:17.091076 14222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:17.095166 14222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:17.095422 14222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:17.096132 14222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:17.097067 14222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:17.097184 14222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:17.097273 14222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:17.097334 14222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:17.107998 14222 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:37887
I20260808 10:37:17.108067 14364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:37887 every 8 connection(s)
I20260808 10:37:17.109091 14222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:37:17.113355 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 14222
I20260808 10:37:17.113471 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:37:17.113749 31306 external_mini_cluster.cc:1428] Setting key 8081d470e7e6efb41ed2374ce17f5789
Aug 08 10:37:17 dist-test-slave-9gcw krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185437, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:37:17.126785 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.111474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:33113 (local address 127.30.146.190:33057)
0808 10:37:17.111641 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:17.111647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:17.112645 (+ 998us) server_negotiation.cc:407] Connection header received
0808 10:37:17.114284 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:17.114287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:17.114371 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:17.114477 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:17.116095 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.116930 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.118773 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.119080 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.119193 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.122210 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:17.122214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:17.122217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:17.122252 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:17.124174 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:17.124835 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:17.124839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:17.124841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:17.124953 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:17.125460 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:17.125464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:17.125466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:17.125702 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:17.125827 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:17.126410 (+ 583us) server_negotiation.cc:299] Negotiation successful
0808 10:37:17.126581 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:37:17.127658 14367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.111892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:33113)
0808 10:37:17.112457 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:17.112507 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:17.114009 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:17.114727 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:17.114737 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:17.115267 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:17.115887 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.115901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.117090 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.117093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.118520 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.118535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.119274 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.119281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.119512 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.120308 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:17.120334 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:17.122011 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:17.124343 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:17.124351 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:17.124377 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:17.124700 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:17.125087 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:17.125090 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:17.125091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:17.125329 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:17.125825 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:17.125834 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:17.126134 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 10:37:17.126347 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 10:37:17.126723 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
I20260808 10:37:17.129220 14365 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33057
I20260808 10:37:17.129509 14365 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:17.130064 14365 heartbeater.cc:511] Master 127.30.146.190:33057 requested a full tablet report, sending...
I20260808 10:37:17.131186 14008 ts_manager.cc:194] Registered new tserver with Master: 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887)
I20260808 10:37:17.131827 14008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:33113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:37:17.148669 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33057
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:17.265668 14371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:17.265945 14371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:17.265988 14371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:17.269985 14371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:17.270045 14371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:17.270104 14371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:37:17.275645 14371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:17.276710 14371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:17.277719 14371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:37:17.288308 14371 server_base.cc:1034] running on GCE node
W20260808 10:37:17.288220 14378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:17.288275 14377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:17.288244 14381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:17.289017 14371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:17.289805 14371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:17.290979 14371 hybrid_clock.cc:648] HybridClock initialized: now 1786185437290954 us; error 37 us; skew 500 ppm
Aug 08 10:37:17 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185437, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:17.294055 14371 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:37:17.295068 14371 webserver.cc:460] Webserver started at http://127.30.146.131:40789/ using document root <none> and password file <none>
I20260808 10:37:17.295706 14371 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:17.295806 14371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:17.296057 14371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:17.298297 14371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "a16be21fe4d44a8d9cf57d007e857379"
format_stamp: "Formatted at 2026-08-08 10:37:17 on dist-test-slave-9gcw"
server_key: "96dc1708d8447461e6572ee8e7b70bd0"
server_key_iv: "c200ede1b7223c2feff64d9fc8907295"
server_key_version: "encryptionkey@0"
I20260808 10:37:17.298892 14371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "a16be21fe4d44a8d9cf57d007e857379"
format_stamp: "Formatted at 2026-08-08 10:37:17 on dist-test-slave-9gcw"
server_key: "96dc1708d8447461e6572ee8e7b70bd0"
server_key_iv: "c200ede1b7223c2feff64d9fc8907295"
server_key_version: "encryptionkey@0"
I20260808 10:37:17.302870 14371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:37:17.305452 14389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:17.306804 14371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:37:17.306953 14371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "a16be21fe4d44a8d9cf57d007e857379"
format_stamp: "Formatted at 2026-08-08 10:37:17 on dist-test-slave-9gcw"
server_key: "96dc1708d8447461e6572ee8e7b70bd0"
server_key_iv: "c200ede1b7223c2feff64d9fc8907295"
server_key_version: "encryptionkey@0"
I20260808 10:37:17.307075 14371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:17.327042 14371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:17.330442 14371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:17.330657 14371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:17.331277 14371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:17.332126 14371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:17.332198 14371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:17.332270 14371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:17.332306 14371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:17.341835 14371 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:38571
I20260808 10:37:17.341862 14526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:38571 every 8 connection(s)
I20260808 10:37:17.342823 14371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:37:17.347040 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 14371
I20260808 10:37:17.347184 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:37:17.347455 31306 external_mini_cluster.cc:1428] Setting key bcf63d22f26e5e4bcc7d04c2cd9d21fa
Aug 08 10:37:17 dist-test-slave-9gcw krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185437, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:17.357486 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.344681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:45623 (local address 127.30.146.190:33057)
0808 10:37:17.344826 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:17.344830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:17.345729 (+ 899us) server_negotiation.cc:407] Connection header received
0808 10:37:17.347168 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:17.347171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:17.347232 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:17.347315 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:17.348755 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.349463 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.350760 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.350983 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.351096 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.353738 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:17.353742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:17.353743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:17.353770 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:17.355260 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:17.355848 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:17.355861 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:17.355863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:17.355946 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:17.356320 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:17.356323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:17.356325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:17.356527 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:17.356640 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:17.357178 (+ 538us) server_negotiation.cc:299] Negotiation successful
0808 10:37:17.357314 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:37:17.358381 14530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.345034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:45623)
0808 10:37:17.345576 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:17.345619 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:17.346934 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:17.347486 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:17.347492 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:17.347952 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:17.348605 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.348622 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.349592 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.349595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.350625 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.350635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.351142 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.351146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.351328 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.351919 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:17.351935 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:17.353552 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:17.355422 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:17.355429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:17.355445 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:17.355724 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:17.356045 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:17.356048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:17.356050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:17.356200 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:17.356637 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:17.356644 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:17.356911 (+ 267us) client_negotiation.cc:769] Sending connection context
0808 10:37:17.357105 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 10:37:17.357518 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":119,"threads_started":1}
I20260808 10:37:17.359650 14527 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33057
I20260808 10:37:17.359879 14527 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:17.360370 14527 heartbeater.cc:511] Master 127.30.146.190:33057 requested a full tablet report, sending...
I20260808 10:37:17.361629 14008 ts_manager.cc:194] Registered new tserver with Master: a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571)
I20260808 10:37:17.362411 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:37:17.362464 14008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:45623
I20260808 10:37:17.372649 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.364916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.30.146.190:33057)
0808 10:37:17.365038 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:17.365042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:17.365212 (+ 170us) server_negotiation.cc:407] Connection header received
0808 10:37:17.365294 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:17.365296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:17.365336 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:17.365405 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:17.366129 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.366660 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.367549 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.367754 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.367838 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.369339 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:17.369342 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:17.369344 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:17.369373 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:17.370808 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:17.371343 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:17.371352 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:17.371353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:17.371442 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:17.371726 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:17.371728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:17.371729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:17.371986 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:17.372099 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:17.372418 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 10:37:17.372527 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:37:17.375856 14008 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:37:17.379012 14008 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:37:17.393864 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.388799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42782 (local address 127.30.146.129:36921)
0808 10:37:17.389230 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:17.389238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:17.389262 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:37:17.389335 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:17.389340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:17.389512 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:17.389658 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:17.389885 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.390689 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.392536 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.393073 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.393258 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.393407 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:17.393588 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:37:17.393699 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":106,"threads_started":1}
I20260808 10:37:17.393896 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.388008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36921 (local address 127.0.0.1:42782)
0808 10:37:17.388518 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:17.388532 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:17.388622 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:17.389652 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:17.389655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:17.389673 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:17.389758 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.389765 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.390816 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.390823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.392395 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.392405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.393254 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.393258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.393380 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.393420 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 10:37:17.393556 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 10:37:17.393754 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":341,"thread_start_us":85,"threads_started":1}
I20260808 10:37:17.393932 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.387622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:37887 (local address 127.0.0.1:54964)
0808 10:37:17.388012 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:17.388067 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:17.388197 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:17.389133 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:17.389160 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:17.389186 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:17.389314 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.389319 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.390891 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.390895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.392362 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.392374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.393389 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.393393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.393489 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.393522 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:37:17.393573 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:37:17.393752 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":83,"thread_start_us":90,"threads_started":1}
I20260808 10:37:17.394517 14541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.387783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54964 (local address 127.30.146.130:37887)
0808 10:37:17.388525 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:17.388538 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:17.388562 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:37:17.388642 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:17.388734 (+ 92us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:17.388958 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:17.389438 (+ 480us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:17.389560 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.390700 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.392493 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.393265 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.394108 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.394239 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:17.394287 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 10:37:17.394379 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":129,"threads_started":1}
I20260808 10:37:17.393931 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.388363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:38571 (local address 127.0.0.1:34136)
0808 10:37:17.388946 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:17.388959 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:17.389049 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:17.389855 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:17.389858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:17.389879 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:17.389979 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.389985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.391455 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.391458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.392228 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.392237 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.393548 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.393554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.393647 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.393668 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:37:17.393733 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:37:17.393790 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":92,"threads_started":1}
I20260808 10:37:17.395466 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.388798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34136 (local address 127.30.146.131:38571)
0808 10:37:17.389231 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:17.389243 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:17.389262 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:37:17.389335 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:17.389340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:17.389512 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:17.389658 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:17.390110 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.391214 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.392352 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.393225 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.393358 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.393491 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:17.393996 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 10:37:17.394454 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":103,"threads_started":1}
I20260808 10:37:17.397204 14295 tablet_service.cc:1467] Processing CreateTablet for tablet ba127bf50a5e447581f2e702ea98c3de (DEFAULT_TABLE table=test-table [id=b4c6af5ce2dd48a49301589be03f81c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:17.397320 14146 tablet_service.cc:1467] Processing CreateTablet for tablet ba127bf50a5e447581f2e702ea98c3de (DEFAULT_TABLE table=test-table [id=b4c6af5ce2dd48a49301589be03f81c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:17.398226 14295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba127bf50a5e447581f2e702ea98c3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:17.397948 14447 tablet_service.cc:1467] Processing CreateTablet for tablet ba127bf50a5e447581f2e702ea98c3de (DEFAULT_TABLE table=test-table [id=b4c6af5ce2dd48a49301589be03f81c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:17.398341 14146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba127bf50a5e447581f2e702ea98c3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:17.398886 14447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba127bf50a5e447581f2e702ea98c3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:17.404861 14547 tablet_bootstrap.cc:492] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: Bootstrap starting.
I20260808 10:37:17.406454 14549 tablet_bootstrap.cc:492] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Bootstrap starting.
I20260808 10:37:17.407425 14547 tablet_bootstrap.cc:654] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:17.408161 14547 log.cc:824] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:17.408185 14549 tablet_bootstrap.cc:654] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:17.408802 14550 tablet_bootstrap.cc:492] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: Bootstrap starting.
I20260808 10:37:17.409026 14549 log.cc:824] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:17.410439 14547 tablet_bootstrap.cc:492] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: No bootstrap required, opened a new log
I20260808 10:37:17.410634 14547 ts_tablet_manager.cc:1397] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:37:17.411068 14549 tablet_bootstrap.cc:492] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: No bootstrap required, opened a new log
I20260808 10:37:17.411257 14549 ts_tablet_manager.cc:1397] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 10:37:17.411481 14550 tablet_bootstrap.cc:654] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:17.412573 14550 log.cc:824] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:17.414245 14550 tablet_bootstrap.cc:492] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: No bootstrap required, opened a new log
I20260808 10:37:17.414438 14550 ts_tablet_manager.cc:1397] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:37:17.414925 14547 raft_consensus.cc:348] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:17.415205 14547 raft_consensus.cc:374] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:17.415264 14547 raft_consensus.cc:729] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119a63ff2f7d497ba776b91b47e67def, State: Initialized, Role: FOLLOWER
I20260808 10:37:17.415740 14547 consensus_queue.cc:260] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:17.415675 14549 raft_consensus.cc:348] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:17.415902 14549 raft_consensus.cc:374] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:17.415949 14549 raft_consensus.cc:729] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9175007187148cd8c27f9344578de42, State: Initialized, Role: FOLLOWER
I20260808 10:37:17.416405 14549 consensus_queue.cc:260] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:17.416759 14547 ts_tablet_manager.cc:1428] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:37:17.417136 14365 heartbeater.cc:503] Master 127.30.146.190:33057 was elected leader, sending a full tablet report...
I20260808 10:37:17.417527 14217 heartbeater.cc:503] Master 127.30.146.190:33057 was elected leader, sending a full tablet report...
I20260808 10:37:17.417378 14549 ts_tablet_manager.cc:1428] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:37:17.418366 14550 raft_consensus.cc:348] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:17.420106 14550 raft_consensus.cc:374] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:17.420181 14550 raft_consensus.cc:729] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16be21fe4d44a8d9cf57d007e857379, State: Initialized, Role: FOLLOWER
I20260808 10:37:17.420598 14550 consensus_queue.cc:260] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:17.421339 14527 heartbeater.cc:503] Master 127.30.146.190:33057 was elected leader, sending a full tablet report...
I20260808 10:37:17.421617 14550 ts_tablet_manager.cc:1428] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:37:17.474344 14555 raft_consensus.cc:482] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:37:17.474612 14555 raft_consensus.cc:504] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:17.476147 14555 leader_election.cc:288] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571), b9175007187148cd8c27f9344578de42 (127.30.146.129:36921)
I20260808 10:37:17.480283 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.476483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:39303 (local address 127.30.146.131:38571)
0808 10:37:17.476652 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:17.476658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:17.476678 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:37:17.476744 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:17.476748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:17.476802 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:17.476884 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:17.477415 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.478073 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.479225 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.479663 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.479768 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.479811 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:17.480060 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 10:37:17.480150 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:37:17.480290 14367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.476379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:38571 (local address 127.30.146.130:39303)
0808 10:37:17.476511 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:17.476530 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:17.476643 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:17.476954 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:17.476957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:17.476977 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:17.477228 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.477238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.478206 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.478209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.479086 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.479096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.479799 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.479802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.479921 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.479947 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:37:17.480024 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:37:17.480095 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":26,"spinlock_wait_cycles":20864}
I20260808 10:37:17.480410 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.476759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:45905 (local address 127.30.146.129:36921)
0808 10:37:17.476889 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:17.476893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:17.477035 (+ 142us) server_negotiation.cc:407] Connection header received
0808 10:37:17.477209 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:17.477212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:17.477272 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:17.477352 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:17.477737 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.478262 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.479360 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.479822 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.479933 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.479983 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:17.480207 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 10:37:17.480282 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:37:17.480647 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.476436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36921 (local address 127.30.146.130:45905)
0808 10:37:17.476924 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:17.476938 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:17.477070 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:17.477415 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:17.477417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:17.477437 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:17.477556 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.477562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.478380 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.478382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.479232 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:17.479242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.479942 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.479944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:17.480064 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.480089 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:37:17.480165 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:37:17.480239 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":18,"spinlock_wait_cycles":4224,"thread_start_us":148,"threads_started":1}
I20260808 10:37:17.481189 14471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba127bf50a5e447581f2e702ea98c3de" candidate_uuid: "119a63ff2f7d497ba776b91b47e67def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16be21fe4d44a8d9cf57d007e857379" is_pre_election: true
I20260808 10:37:17.481240 14166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba127bf50a5e447581f2e702ea98c3de" candidate_uuid: "119a63ff2f7d497ba776b91b47e67def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9175007187148cd8c27f9344578de42" is_pre_election: true
I20260808 10:37:17.481514 14471 raft_consensus.cc:2454] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 119a63ff2f7d497ba776b91b47e67def in term 0.
I20260808 10:37:17.481535 14166 raft_consensus.cc:2454] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 119a63ff2f7d497ba776b91b47e67def in term 0.
I20260808 10:37:17.482059 14244 leader_election.cc:302] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 119a63ff2f7d497ba776b91b47e67def, a16be21fe4d44a8d9cf57d007e857379, b9175007187148cd8c27f9344578de42; no voters:
I20260808 10:37:17.482349 14555 raft_consensus.cc:2781] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:37:17.482437 14555 raft_consensus.cc:482] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:37:17.482482 14555 raft_consensus.cc:3037] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:17.483763 14555 raft_consensus.cc:504] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:17.484257 14555 leader_election.cc:288] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [CANDIDATE]: Term 1 election: Requested vote from peers a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571), b9175007187148cd8c27f9344578de42 (127.30.146.129:36921)
I20260808 10:37:17.484733 14471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba127bf50a5e447581f2e702ea98c3de" candidate_uuid: "119a63ff2f7d497ba776b91b47e67def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16be21fe4d44a8d9cf57d007e857379"
I20260808 10:37:17.484913 14166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba127bf50a5e447581f2e702ea98c3de" candidate_uuid: "119a63ff2f7d497ba776b91b47e67def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9175007187148cd8c27f9344578de42"
I20260808 10:37:17.484916 14471 raft_consensus.cc:3037] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:17.485042 14166 raft_consensus.cc:3037] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:17.486445 14471 raft_consensus.cc:2454] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 119a63ff2f7d497ba776b91b47e67def in term 1.
I20260808 10:37:17.486500 14166 raft_consensus.cc:2454] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 119a63ff2f7d497ba776b91b47e67def in term 1.
I20260808 10:37:17.486820 14242 leader_election.cc:302] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119a63ff2f7d497ba776b91b47e67def, a16be21fe4d44a8d9cf57d007e857379; no voters:
I20260808 10:37:17.486984 14555 raft_consensus.cc:2781] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:37:17.487265 14555 raft_consensus.cc:686] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 1 LEADER]: Becoming Leader. State: Replica: 119a63ff2f7d497ba776b91b47e67def, State: Running, Role: LEADER
I20260808 10:37:17.487627 14555 consensus_queue.cc:237] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:17.491016 14010 catalog_manager.cc:5495] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def reported cstate change: term changed from 0 to 1, leader changed from <none> to 119a63ff2f7d497ba776b91b47e67def (127.30.146.130). New cstate: current_term: 1 leader_uuid: "119a63ff2f7d497ba776b91b47e67def" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } health_report { overall_health: UNKNOWN } } }
I20260808 10:37:17.517910 14541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.514096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54976 (local address 127.30.146.130:37887)
0808 10:37:17.514248 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:17.514252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:17.514266 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:37:17.514344 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:17.514346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:17.514401 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:17.514496 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:37:17.514765 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.515326 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.516184 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:17.516425 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:17.516529 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:17.516871 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:37:17.517431 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:37:17.517535 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:17.517679 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 10:37:17.517786 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:37:17.524101 14471 raft_consensus.cc:1261] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379 [term 1 FOLLOWER]: Refusing update from remote peer 119a63ff2f7d497ba776b91b47e67def: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:17.524215 14166 raft_consensus.cc:1261] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Refusing update from remote peer 119a63ff2f7d497ba776b91b47e67def: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:17.524942 14559 consensus_queue.cc:1035] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:17.525166 14555 consensus_queue.cc:1035] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:17.543104 14008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:37:17.543296 14008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:37:17.546093 14008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Sending DeleteTablet for 3 replicas of tablet ba127bf50a5e447581f2e702ea98c3de
I20260808 10:37:17.547025 14447 tablet_service.cc:1514] Processing DeleteTablet for tablet ba127bf50a5e447581f2e702ea98c3de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:17 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:34136
I20260808 10:37:17.547051 14146 tablet_service.cc:1514] Processing DeleteTablet for tablet ba127bf50a5e447581f2e702ea98c3de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:17 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:42782
I20260808 10:37:17.547598 14574 tablet_replica.cc:330] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: stopping tablet replica
I20260808 10:37:17.547824 14295 tablet_service.cc:1514] Processing DeleteTablet for tablet ba127bf50a5e447581f2e702ea98c3de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:17 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:54964
I20260808 10:37:17.547962 14574 raft_consensus.cc:2229] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:17.548280 14574 raft_consensus.cc:2258] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:17.548761 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 14074
I20260808 10:37:17.551285 14575 tablet_replica.cc:330] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: stopping tablet replica
I20260808 10:37:17.551644 14575 raft_consensus.cc:2229] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:37:17.552054 14575 raft_consensus.cc:2258] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:17.552558 14573 tablet_replica.cc:330] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: stopping tablet replica
I20260808 10:37:17.552919 14573 raft_consensus.cc:2229] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:17.553318 14573 raft_consensus.cc:2258] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:17.554648 14575 ts_tablet_manager.cc:1905] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:17.555023 14573 ts_tablet_manager.cc:1905] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:17.557705 14575 ts_tablet_manager.cc:1918] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:17.557893 14575 log.cc:1196] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/ba127bf50a5e447581f2e702ea98c3de
I20260808 10:37:17.558329 14575 ts_tablet_manager.cc:1939] T ba127bf50a5e447581f2e702ea98c3de P 119a63ff2f7d497ba776b91b47e67def: Deleting consensus metadata
I20260808 10:37:17.558722 14573 ts_tablet_manager.cc:1918] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:17.558845 14573 log.cc:1196] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/ba127bf50a5e447581f2e702ea98c3de
W20260808 10:37:17.559180 13995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:36921 (error 108)
I20260808 10:37:17.559255 14573 ts_tablet_manager.cc:1939] T ba127bf50a5e447581f2e702ea98c3de P a16be21fe4d44a8d9cf57d007e857379: Deleting consensus metadata
I20260808 10:37:17.559929 13994 catalog_manager.cc:4841] TS 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887): tablet ba127bf50a5e447581f2e702ea98c3de (table test-table [id=b4c6af5ce2dd48a49301589be03f81c2]) successfully deleted
I20260808 10:37:17.560245 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 14222
I20260808 10:37:17.564327 13994 catalog_manager.cc:4841] TS a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571): tablet ba127bf50a5e447581f2e702ea98c3de (table test-table [id=b4c6af5ce2dd48a49301589be03f81c2]) successfully deleted
I20260808 10:37:17.566416 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:17.566046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36921 (local address 127.0.0.1:42796)
0808 10:37:17.566241 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:17.566319 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:36921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260808 10:37:17.566669 13995 catalog_manager.cc:4584] TS b9175007187148cd8c27f9344578de42 (127.30.146.129:36921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba127bf50a5e447581f2e702ea98c3de: Network error: Client connection negotiation failed: client connection to 127.30.146.129:36921: connect: Connection refused (error 111)
I20260808 10:37:17.573210 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 14371
I20260808 10:37:17.582945 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 13975
I20260808 10:37:17.591950 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33057
--webserver_interface=127.30.146.190
--webserver_port=38113
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:33057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:17.711946 14578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:17.712280 14578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:17.712364 14578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:17.716511 14578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:17.716610 14578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:17.716645 14578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:17.716687 14578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:17.716719 14578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:17.722275 14578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:33057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=38113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:17.723394 14578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:17.724478 14578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:17.740833 14586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:17.740854 14585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:17.740840 14588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:17.741287 14578 server_base.cc:1034] running on GCE node
I20260808 10:37:17.742020 14578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:17.743283 14578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:17.744485 14578 hybrid_clock.cc:648] HybridClock initialized: now 1786185437744467 us; error 37 us; skew 500 ppm
Aug 08 10:37:17 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185437, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:17.747833 14578 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:37:17.748881 14578 webserver.cc:460] Webserver started at http://127.30.146.190:38113/ using document root <none> and password file <none>
I20260808 10:37:17.749557 14578 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:17.749678 14578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:17.754069 14578 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:37:17.756603 14595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:17.758419 14578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:37:17.758625 14578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "c2b184d8c0674d81b9aac53d8717d8b0"
format_stamp: "Formatted at 2026-08-08 10:37:16 on dist-test-slave-9gcw"
server_key: "8ab563a642febd3aa3e0abee657c30d0"
server_key_iv: "08c23d8c8c87719a20752ab2d6836bff"
server_key_version: "encryptionkey@0"
I20260808 10:37:17.759359 14578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:17.788897 14578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:17.792709 14578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:17.792974 14578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:17.801477 14578 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:33057
I20260808 10:37:17.801788 14653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:33057 every 8 connection(s)
I20260808 10:37:17.802762 14578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:37:17.807256 14654 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:37:17.808974 14654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Bootstrap starting.
I20260808 10:37:17.810595 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 14578
I20260808 10:37:17.811075 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:36921
--local_ip_for_outbound_sockets=127.30.146.129
--tserver_master_addrs=127.30.146.190:33057
--webserver_port=45269
--webserver_interface=127.30.146.129
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:37:17.827695 14654 log.cc:824] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:17.836961 14654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:37:17.837446 14654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Bootstrap complete.
I20260808 10:37:17.842077 14654 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:17.842540 14654 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b184d8c0674d81b9aac53d8717d8b0, State: Initialized, Role: FOLLOWER
I20260808 10:37:17.843046 14654 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:17.843196 14654 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:37:17.843303 14654 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:37:17.843420 14654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:37:17.845562 14654 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:17.845947 14654 leader_election.cc:302] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2b184d8c0674d81b9aac53d8717d8b0; no voters:
I20260808 10:37:17.846367 14654 leader_election.cc:288] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:37:17.846503 14659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:37:17.846776 14659 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 2 LEADER]: Becoming Leader. State: Replica: c2b184d8c0674d81b9aac53d8717d8b0, State: Running, Role: LEADER
I20260808 10:37:17.847101 14659 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:17.847700 14654 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:17.848587 14660 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } } }
I20260808 10:37:17.848737 14660 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:17.848956 14660 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2b184d8c0674d81b9aac53d8717d8b0. Latest consensus state: current_term: 2 leader_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } } }
I20260808 10:37:17.849038 14660 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:17.849777 14669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:37:17.855103 14669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:37:17.855904 14669 catalog_manager.cc:1259] Loaded cluster ID: a7f1ea174d8d4918a092746bdb34cdd0
I20260808 10:37:17.855978 14669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:37:17.857553 14669 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 10:37:17.862310 14669 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 10:37:17.868932 14669 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:37:17.869860 14669 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Loaded TSK: 0
I20260808 10:37:17.871104 14669 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:37:17.939911 14656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:17.940210 14656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:17.940294 14656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:17.944499 14656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:17.944595 14656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:17.944698 14656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:37:17.950398 14656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:36921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=45269
--tserver_master_addrs=127.30.146.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:17.951519 14656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:17.952574 14656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:17.962565 14686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:17.962692 14684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:17.962581 14683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:17.963094 14656 server_base.cc:1034] running on GCE node
I20260808 10:37:17.963620 14656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:17.964469 14656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:17.965685 14656 hybrid_clock.cc:648] HybridClock initialized: now 1786185437965658 us; error 35 us; skew 500 ppm
Aug 08 10:37:17 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185437, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:17.969064 14656 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:37:17.970118 14656 webserver.cc:460] Webserver started at http://127.30.146.129:45269/ using document root <none> and password file <none>
I20260808 10:37:17.970718 14656 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:17.970832 14656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:17.975061 14656 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 10:37:17.977377 14693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:17.978756 14656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:37:17.978906 14656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "b9175007187148cd8c27f9344578de42"
format_stamp: "Formatted at 2026-08-08 10:37:16 on dist-test-slave-9gcw"
server_key: "ee5a3d3262304e6e058fcf5437ee1f76"
server_key_iv: "71a35e03f578d7f2c4c2b3d5d184ac1a"
server_key_version: "encryptionkey@0"
I20260808 10:37:17.979475 14656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:18.000334 14656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:18.003849 14656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:18.004092 14656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:18.004822 14656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:18.006100 14702 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:37:18.010758 14656 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:37:18.010848 14656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 10:37:18.010941 14656 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:37:18.012720 14656 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:37:18.012799 14656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 10:37:18.013028 14702 tablet_bootstrap.cc:492] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Bootstrap starting.
I20260808 10:37:18.023756 14656 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:36921
I20260808 10:37:18.023880 14821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:36921 every 8 connection(s)
I20260808 10:37:18.024751 14656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:37:18.030156 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 14656
I20260808 10:37:18.030798 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:37887
--local_ip_for_outbound_sockets=127.30.146.130
--tserver_master_addrs=127.30.146.190:33057
--webserver_port=35787
--webserver_interface=127.30.146.130
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:37:18.036587 14702 log.cc:824] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:37:18 dist-test-slave-9gcw krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185437, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:18.045855 14702 tablet_bootstrap.cc:492] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:37:18.046540 14702 tablet_bootstrap.cc:492] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Bootstrap complete.
I20260808 10:37:18.046785 14826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.026992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:47783)
0808 10:37:18.027545 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:18.027586 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:18.028430 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:18.029615 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:18.029623 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:18.030074 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:18.030591 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.030607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.032056 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.032059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.033093 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.033106 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.034392 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.034399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.034676 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.035498 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:18.035524 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:18.037846 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:18.042954 (+ 5108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:18.042963 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:18.042981 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:18.043393 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:18.043928 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:18.043934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:18.043938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:18.044136 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:18.044940 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:18.044947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:18.045274 (+ 327us) client_negotiation.cc:769] Sending connection context
0808 10:37:18.045450 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 10:37:18.045799 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":160,"threads_started":1}
I20260808 10:37:18.048149 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.026876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:47783 (local address 127.30.146.190:33057)
0808 10:37:18.027262 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.027278 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.027709 (+ 431us) server_negotiation.cc:407] Connection header received
0808 10:37:18.028856 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.028866 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.029221 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.029567 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:18.030780 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.031883 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.033279 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.034209 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.034404 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.038147 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:18.038154 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:18.038168 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:18.038201 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:18.042768 (+ 4567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:18.043571 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:18.043578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:18.043587 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:18.043793 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:18.044292 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:18.044298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:18.044302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:18.044804 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:18.046190 (+ 1386us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.046365 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.046792 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":120,"threads_started":1}
I20260808 10:37:18.048367 14702 ts_tablet_manager.cc:1397] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Time spent bootstrapping tablet: real 0.036s user 0.024s sys 0.009s
I20260808 10:37:18.050704 14823 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33057
I20260808 10:37:18.050956 14823 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:18.051549 14823 heartbeater.cc:511] Master 127.30.146.190:33057 requested a full tablet report, sending...
I20260808 10:37:18.051517 14702 raft_consensus.cc:348] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:18.052141 14702 raft_consensus.cc:729] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9175007187148cd8c27f9344578de42, State: Initialized, Role: FOLLOWER
I20260808 10:37:18.052606 14702 consensus_queue.cc:260] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:18.053766 14702 ts_tablet_manager.cc:1428] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:37:18.055859 14614 ts_manager.cc:194] Registered new tserver with Master: b9175007187148cd8c27f9344578de42 (127.30.146.129:36921)
I20260808 10:37:18.059544 14614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:47783
I20260808 10:37:18.062215 14823 heartbeater.cc:503] Master 127.30.146.190:33057 was elected leader, sending a full tablet report...
Aug 08 10:37:18 dist-test-slave-9gcw krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185437, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:37:18.071051 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.058887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36921 (local address 127.0.0.1:42804)
0808 10:37:18.059688 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:18.059737 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:18.059882 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:18.060450 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:18.060457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:18.060915 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:18.061060 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.061083 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.062527 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.062530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.063419 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.063430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.064101 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.064106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.064231 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.064916 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:18.064930 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:18.066673 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:18.068886 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:18.068896 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:18.068900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:18.069230 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:18.069678 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:18.069680 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:18.069682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:18.069778 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:18.070393 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:18.070399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:18.070515 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 10:37:18.070719 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 10:37:18.070906 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":133,"threads_started":1}
I20260808 10:37:18.071189 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.058984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42804 (local address 127.30.146.129:36921)
0808 10:37:18.059400 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.059411 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.059849 (+ 438us) server_negotiation.cc:407] Connection header received
0808 10:37:18.060043 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.060049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.060264 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.060422 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:18.061243 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.062395 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.063584 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.063950 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.064092 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.066850 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:18.066860 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:18.066872 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:18.066899 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:18.068765 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:18.069364 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:18.069373 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:18.069377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:18.069483 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:18.069906 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:18.069911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:18.069914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:18.070250 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:18.070385 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.070848 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.071024 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":120,"threads_started":1}
I20260808 10:37:18.072268 14748 tablet_service.cc:1514] Processing DeleteTablet for tablet ba127bf50a5e447581f2e702ea98c3de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:17 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:42804
I20260808 10:37:18.073437 14832 tablet_replica.cc:330] stopping tablet replica
I20260808 10:37:18.073755 14832 raft_consensus.cc:2229] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:18.074014 14832 raft_consensus.cc:2258] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:18.075026 14832 ts_tablet_manager.cc:1905] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:18.078709 14832 ts_tablet_manager.cc:1918] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:18.078828 14832 log.cc:1196] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/ba127bf50a5e447581f2e702ea98c3de
I20260808 10:37:18.079106 14832 ts_tablet_manager.cc:1939] T ba127bf50a5e447581f2e702ea98c3de P b9175007187148cd8c27f9344578de42: Deleting consensus metadata
I20260808 10:37:18.080830 14598 catalog_manager.cc:4841] TS b9175007187148cd8c27f9344578de42 (127.30.146.129:36921): tablet ba127bf50a5e447581f2e702ea98c3de (table test-table [id=b4c6af5ce2dd48a49301589be03f81c2]) successfully deleted
W20260808 10:37:18.156445 14827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:18.156726 14827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:18.156818 14827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:18.161268 14827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:18.161330 14827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:18.161417 14827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:37:18.167078 14827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:37887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=35787
--tserver_master_addrs=127.30.146.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:18.168164 14827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:18.169301 14827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:18.179802 14841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:18.179812 14838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:18.179807 14839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:18.179850 14827 server_base.cc:1034] running on GCE node
I20260808 10:37:18.180459 14827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:18.181234 14827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:18.182452 14827 hybrid_clock.cc:648] HybridClock initialized: now 1786185438182429 us; error 39 us; skew 500 ppm
Aug 08 10:37:18 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185438, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:18.185696 14827 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:37:18.186584 14827 webserver.cc:460] Webserver started at http://127.30.146.130:35787/ using document root <none> and password file <none>
I20260808 10:37:18.187124 14827 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:18.187203 14827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:18.191269 14827 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:37:18.193377 14849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:18.194597 14827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:37:18.194698 14827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "119a63ff2f7d497ba776b91b47e67def"
format_stamp: "Formatted at 2026-08-08 10:37:17 on dist-test-slave-9gcw"
server_key: "aaabfe5acdccc59e34f81d66cb557da3"
server_key_iv: "4a4adf50e02a8aec1205e4da3e0bd3e3"
server_key_version: "encryptionkey@0"
I20260808 10:37:18.195286 14827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:18.214084 14827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:18.217679 14827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:18.217903 14827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:18.218549 14827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:18.219504 14827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:18.219560 14827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:18.219614 14827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:18.219678 14827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:18.229535 14827 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:37887
I20260808 10:37:18.229568 14975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:37887 every 8 connection(s)
I20260808 10:37:18.230613 14827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:37:18.239646 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 14827
I20260808 10:37:18.240118 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:38571
--local_ip_for_outbound_sockets=127.30.146.131
--tserver_master_addrs=127.30.146.190:33057
--webserver_port=40789
--webserver_interface=127.30.146.131
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:37:18 dist-test-slave-9gcw krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185438, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:18.246289 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.232612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:42739 (local address 127.30.146.190:33057)
0808 10:37:18.232827 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.232831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.233825 (+ 994us) server_negotiation.cc:407] Connection header received
0808 10:37:18.235173 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.235176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.235228 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.235314 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:18.236605 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.237266 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.238615 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.238851 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.238945 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.242055 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:18.242060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:18.242063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:18.242101 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:18.243739 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:18.244325 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:18.244329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:18.244331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:18.244423 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:18.244896 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:18.244901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:18.244903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:18.245106 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:18.245265 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.245990 (+ 725us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.246112 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260808 10:37:18.247514 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.233073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:42739)
0808 10:37:18.233588 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:18.233698 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:18.234951 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:18.235493 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:18.235499 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:18.235961 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:18.236445 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.236459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.237411 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.237414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.238471 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.238483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.239002 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.239005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.239164 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.239901 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:18.239919 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:18.241858 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:18.243887 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:18.243893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:18.243908 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:18.244206 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:18.244539 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:18.244542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:18.244544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:18.244786 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:18.245314 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:18.245324 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:18.245692 (+ 368us) client_negotiation.cc:769] Sending connection context
0808 10:37:18.245936 (+ 244us) client_negotiation.cc:240] Negotiation successful
0808 10:37:18.246320 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":114,"threads_started":1}
I20260808 10:37:18.248905 14976 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33057
I20260808 10:37:18.249308 14976 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:18.249850 14976 heartbeater.cc:511] Master 127.30.146.190:33057 requested a full tablet report, sending...
I20260808 10:37:18.251011 14614 ts_manager.cc:194] Registered new tserver with Master: 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887)
I20260808 10:37:18.251653 14614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:42739
W20260808 10:37:18.358767 14979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:18.359030 14979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:18.359073 14979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:18.363245 14979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:18.363313 14979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:18.363412 14979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:37:18.369247 14979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:38571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=40789
--tserver_master_addrs=127.30.146.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:18.370397 14979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:18.371446 14979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:37:18.381431 14979 server_base.cc:1034] running on GCE node
W20260808 10:37:18.381422 14988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:18.381417 14986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:18.381879 14984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:18.382300 14979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:18.383057 14979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:18.384282 14979 hybrid_clock.cc:648] HybridClock initialized: now 1786185438384264 us; error 42 us; skew 500 ppm
Aug 08 10:37:18 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185438, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:18.387624 14979 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:37:18.388574 14979 webserver.cc:460] Webserver started at http://127.30.146.131:40789/ using document root <none> and password file <none>
I20260808 10:37:18.389273 14979 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:18.389387 14979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:18.393530 14979 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:37:18.395686 14996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:18.396946 14979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:37:18.397080 14979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "a16be21fe4d44a8d9cf57d007e857379"
format_stamp: "Formatted at 2026-08-08 10:37:17 on dist-test-slave-9gcw"
server_key: "96dc1708d8447461e6572ee8e7b70bd0"
server_key_iv: "c200ede1b7223c2feff64d9fc8907295"
server_key_version: "encryptionkey@0"
I20260808 10:37:18.397745 14979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:18.436276 14979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:18.439677 14979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:18.439891 14979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:18.440518 14979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:18.441553 14979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:18.441627 14979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:18.441699 14979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:18.441749 14979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:18.452127 14979 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:38571
I20260808 10:37:18.452167 15119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:38571 every 8 connection(s)
I20260808 10:37:18.453186 14979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:37:18.458570 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 14979
Aug 08 10:37:18 dist-test-slave-9gcw krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185438, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:18.467752 15123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.459286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57680 (local address 127.30.146.190:33057)
0808 10:37:18.459563 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.459566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.459720 (+ 154us) server_negotiation.cc:407] Connection header received
0808 10:37:18.459824 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.459826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.459866 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.459963 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:18.460756 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.461308 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.462169 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.462402 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.462508 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.464094 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:18.464098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:18.464101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:18.464152 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:18.466029 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:18.466476 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:18.466481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:18.466483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:18.466573 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:18.466859 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:18.466863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:18.466865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:18.467058 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:18.467290 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.467427 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.467553 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
I20260808 10:37:18.468319 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.455014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:44635 (local address 127.30.146.190:33057)
0808 10:37:18.455153 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.455161 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.456078 (+ 917us) server_negotiation.cc:407] Connection header received
0808 10:37:18.457565 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.457569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.457630 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.457714 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:18.459017 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.459763 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.460910 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.461199 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.461290 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.464312 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:18.464315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:18.464318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:18.464356 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:18.466117 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:18.466652 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:18.466655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:18.466657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:18.466743 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:18.467105 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:18.467110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:18.467112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:18.467292 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:18.467405 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.468021 (+ 616us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.468140 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":52}
I20260808 10:37:18.469483 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.455425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:44635)
0808 10:37:18.455925 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:18.455970 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:18.457348 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:18.457907 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:18.457914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:18.458313 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:18.458859 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.458875 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.459891 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.459899 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.460776 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.460785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.461355 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.461359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.461538 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.462179 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:18.462196 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:18.464138 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:18.466247 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:18.466258 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:18.466270 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:18.466541 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:18.466848 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:18.466851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:18.466853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:18.467001 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:18.467406 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:18.467414 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:18.467725 (+ 311us) client_negotiation.cc:769] Sending connection context
0808 10:37:18.467971 (+ 246us) client_negotiation.cc:240] Negotiation successful
0808 10:37:18.468518 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":107,"threads_started":1}
I20260808 10:37:18.470564 15120 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33057
I20260808 10:37:18.470860 15120 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:18.471364 15120 heartbeater.cc:511] Master 127.30.146.190:33057 requested a full tablet report, sending...
I20260808 10:37:18.472369 14614 ts_manager.cc:194] Registered new tserver with Master: a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571)
I20260808 10:37:18.472905 14614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:44635
I20260808 10:37:18.481630 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:37:18.486387 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.482321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.30.146.190:33057)
0808 10:37:18.482470 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.482476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.482657 (+ 181us) server_negotiation.cc:407] Connection header received
0808 10:37:18.482887 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.482889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.483000 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.483134 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:37:18.483414 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.484000 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.484894 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.485111 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.485253 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.485580 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:37:18.485985 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:37:18.486122 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.486170 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.486225 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:37:18.487061 14614 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:37:18.490414 14614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:37:18.498693 14748 tablet_service.cc:1467] Processing CreateTablet for tablet e50eaf4d26b4492983b3915e71691955 (DEFAULT_TABLE table=test-table [id=470eb8c3eb014bfbbd338f3c216e6771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:18.499498 14748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e50eaf4d26b4492983b3915e71691955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:18.502553 14702 tablet_bootstrap.cc:492] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Bootstrap starting.
I20260808 10:37:18.503568 14702 tablet_bootstrap.cc:654] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:18.504134 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.497634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:38571 (local address 127.0.0.1:34146)
0808 10:37:18.497759 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:18.497778 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:18.497869 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:18.499199 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:18.499205 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:18.499228 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:18.499386 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.499395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.501178 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.501182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.502564 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.502576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.503666 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.503671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.503769 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.503831 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 10:37:18.503901 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:37:18.503989 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":3}
I20260808 10:37:18.504457 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.497975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.30.146.131:38571)
0808 10:37:18.498553 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.498566 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.498592 (+ 26us) server_negotiation.cc:407] Connection header received
0808 10:37:18.498751 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.498758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.498992 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.499539 (+ 547us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:18.499735 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.500995 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.502927 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.503513 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.503782 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.503931 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.503985 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.504078 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":202,"threads_started":1}
I20260808 10:37:18.504526 15131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.497933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:37887 (local address 127.0.0.1:54980)
0808 10:37:18.498638 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:18.498675 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:18.498777 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:18.499318 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:18.499321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:18.499340 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:18.500015 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.500023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.501580 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.501585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.502633 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.502645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.504073 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.504076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.504185 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.504220 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 10:37:18.504293 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:37:18.504370 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":491,"thread_start_us":124,"threads_started":1}
I20260808 10:37:18.504989 15133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.498222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54980 (local address 127.30.146.130:37887)
0808 10:37:18.498553 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.498566 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.498750 (+ 184us) server_negotiation.cc:407] Connection header received
0808 10:37:18.498894 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.498898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.499099 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.499409 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:18.500160 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.501315 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.502796 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.503662 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.503820 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.503995 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.504722 (+ 727us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.504830 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":110,"threads_started":1}
I20260808 10:37:18.505528 14702 tablet_bootstrap.cc:492] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: No bootstrap required, opened a new log
I20260808 10:37:18.505630 14702 ts_tablet_manager.cc:1397] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:37:18.506108 14702 raft_consensus.cc:348] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:18.506248 14702 raft_consensus.cc:374] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:18.506292 14702 raft_consensus.cc:729] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9175007187148cd8c27f9344578de42, State: Initialized, Role: FOLLOWER
I20260808 10:37:18.506458 14702 consensus_queue.cc:260] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:18.506735 14702 ts_tablet_manager.cc:1428] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:37:18.508023 15047 tablet_service.cc:1467] Processing CreateTablet for tablet e50eaf4d26b4492983b3915e71691955 (DEFAULT_TABLE table=test-table [id=470eb8c3eb014bfbbd338f3c216e6771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:18.508002 14905 tablet_service.cc:1467] Processing CreateTablet for tablet e50eaf4d26b4492983b3915e71691955 (DEFAULT_TABLE table=test-table [id=470eb8c3eb014bfbbd338f3c216e6771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:18.509189 14905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e50eaf4d26b4492983b3915e71691955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:18.509195 15047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e50eaf4d26b4492983b3915e71691955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:18.516170 15136 tablet_bootstrap.cc:492] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: Bootstrap starting.
I20260808 10:37:18.516150 15135 tablet_bootstrap.cc:492] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: Bootstrap starting.
I20260808 10:37:18.518790 15135 tablet_bootstrap.cc:654] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:18.518782 15136 tablet_bootstrap.cc:654] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:18.519532 15136 log.cc:824] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:18.519534 15135 log.cc:824] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:18.521757 15135 tablet_bootstrap.cc:492] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: No bootstrap required, opened a new log
I20260808 10:37:18.521835 15136 tablet_bootstrap.cc:492] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: No bootstrap required, opened a new log
I20260808 10:37:18.521966 15135 ts_tablet_manager.cc:1397] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260808 10:37:18.521986 15136 ts_tablet_manager.cc:1397] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
W20260808 10:37:18.526165 14824 tablet.cc:2369] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:37:18.526057 15136 raft_consensus.cc:348] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:18.526057 15135 raft_consensus.cc:348] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:18.526333 15135 raft_consensus.cc:374] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:18.526333 15136 raft_consensus.cc:374] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:18.526397 15135 raft_consensus.cc:729] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16be21fe4d44a8d9cf57d007e857379, State: Initialized, Role: FOLLOWER
I20260808 10:37:18.526397 15136 raft_consensus.cc:729] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119a63ff2f7d497ba776b91b47e67def, State: Initialized, Role: FOLLOWER
I20260808 10:37:18.526918 15136 consensus_queue.cc:260] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:18.526918 15135 consensus_queue.cc:260] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:18.527693 15120 heartbeater.cc:503] Master 127.30.146.190:33057 was elected leader, sending a full tablet report...
I20260808 10:37:18.527840 14976 heartbeater.cc:503] Master 127.30.146.190:33057 was elected leader, sending a full tablet report...
I20260808 10:37:18.527989 15135 ts_tablet_manager.cc:1428] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:37:18.527989 15136 ts_tablet_manager.cc:1428] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 10:37:18.704216 15121 tablet.cc:2369] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:37:18.713732 15140 raft_consensus.cc:482] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:37:18.713989 15140 raft_consensus.cc:504] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:18.715473 15140 leader_election.cc:288] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571), b9175007187148cd8c27f9344578de42 (127.30.146.129:36921)
I20260808 10:37:18.721200 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.716059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:52799 (local address 127.30.146.131:38571)
0808 10:37:18.716256 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.716262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.716379 (+ 117us) server_negotiation.cc:407] Connection header received
0808 10:37:18.716575 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.716581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.716653 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.716749 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:18.717084 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.718020 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.719813 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.720488 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.720737 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.720793 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.720948 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.721021 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 10:37:18.721230 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.716059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:48167 (local address 127.30.146.129:36921)
0808 10:37:18.716419 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.716426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.716514 (+ 88us) server_negotiation.cc:407] Connection header received
0808 10:37:18.716715 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.716720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.716795 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.716914 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:18.717266 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.718083 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.719813 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.720558 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.720685 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.720858 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.720923 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.721006 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":99,"threads_started":1}
I20260808 10:37:18.721236 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.715947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:38571 (local address 127.30.146.130:52799)
0808 10:37:18.716268 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:18.716287 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:18.716422 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:18.716779 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:18.716782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:18.716799 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:18.716950 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.716958 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.718216 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.718220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.719656 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.719669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.720653 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.720657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.720778 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.720815 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 10:37:18.720903 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:37:18.721018 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":54}
I20260808 10:37:18.721244 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.715947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36921 (local address 127.30.146.130:48167)
0808 10:37:18.716403 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:18.716417 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:18.716523 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:18.716953 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:18.716956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:18.716972 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:18.717119 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.717125 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.718216 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.718220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.719656 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:18.719669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.720717 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.720720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:18.720815 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.720837 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:37:18.720903 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:37:18.721018 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":173,"thread_start_us":112,"threads_started":1}
I20260808 10:37:18.722052 14769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e50eaf4d26b4492983b3915e71691955" candidate_uuid: "119a63ff2f7d497ba776b91b47e67def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9175007187148cd8c27f9344578de42" is_pre_election: true
I20260808 10:37:18.722440 14769 raft_consensus.cc:2454] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 119a63ff2f7d497ba776b91b47e67def in term 0.
I20260808 10:37:18.722589 15068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e50eaf4d26b4492983b3915e71691955" candidate_uuid: "119a63ff2f7d497ba776b91b47e67def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16be21fe4d44a8d9cf57d007e857379" is_pre_election: true
I20260808 10:37:18.723018 14853 leader_election.cc:302] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119a63ff2f7d497ba776b91b47e67def, b9175007187148cd8c27f9344578de42; no voters:
I20260808 10:37:18.723074 15068 raft_consensus.cc:2454] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 119a63ff2f7d497ba776b91b47e67def in term 0.
I20260808 10:37:18.723435 15140 raft_consensus.cc:2781] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:37:18.723573 15140 raft_consensus.cc:482] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:37:18.723654 15140 raft_consensus.cc:3037] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:18.724892 15140 raft_consensus.cc:504] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:18.725354 15140 leader_election.cc:288] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [CANDIDATE]: Term 1 election: Requested vote from peers a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571), b9175007187148cd8c27f9344578de42 (127.30.146.129:36921)
I20260808 10:37:18.725809 14769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e50eaf4d26b4492983b3915e71691955" candidate_uuid: "119a63ff2f7d497ba776b91b47e67def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9175007187148cd8c27f9344578de42"
I20260808 10:37:18.725868 15068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e50eaf4d26b4492983b3915e71691955" candidate_uuid: "119a63ff2f7d497ba776b91b47e67def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16be21fe4d44a8d9cf57d007e857379"
I20260808 10:37:18.725970 14769 raft_consensus.cc:3037] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:18.726009 15068 raft_consensus.cc:3037] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:18.727171 14769 raft_consensus.cc:2454] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 119a63ff2f7d497ba776b91b47e67def in term 1.
I20260808 10:37:18.727267 15068 raft_consensus.cc:2454] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 119a63ff2f7d497ba776b91b47e67def in term 1.
I20260808 10:37:18.727499 14853 leader_election.cc:302] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119a63ff2f7d497ba776b91b47e67def, b9175007187148cd8c27f9344578de42; no voters:
I20260808 10:37:18.727707 15140 raft_consensus.cc:2781] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:37:18.727998 15140 raft_consensus.cc:686] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 1 LEADER]: Becoming Leader. State: Replica: 119a63ff2f7d497ba776b91b47e67def, State: Running, Role: LEADER
I20260808 10:37:18.728399 15140 consensus_queue.cc:237] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
W20260808 10:37:18.731861 14977 tablet.cc:2369] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:37:18.731948 14614 catalog_manager.cc:5495] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def reported cstate change: term changed from 0 to 1, leader changed from <none> to 119a63ff2f7d497ba776b91b47e67def (127.30.146.130). New cstate: current_term: 1 leader_uuid: "119a63ff2f7d497ba776b91b47e67def" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } health_report { overall_health: UNKNOWN } } }
I20260808 10:37:18.800413 15133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.796584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54988 (local address 127.30.146.130:37887)
0808 10:37:18.796723 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:18.796727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:18.796769 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:37:18.796854 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:18.796856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:18.796917 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:18.797019 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:37:18.797303 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.797838 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.798691 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:18.798970 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:18.799053 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:18.799407 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:37:18.799891 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:37:18.799993 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:18.800154 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 10:37:18.800239 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:37:18.806846 15068 raft_consensus.cc:1261] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379 [term 1 FOLLOWER]: Refusing update from remote peer 119a63ff2f7d497ba776b91b47e67def: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:18.806852 14769 raft_consensus.cc:1261] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Refusing update from remote peer 119a63ff2f7d497ba776b91b47e67def: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:18.807603 15140 consensus_queue.cc:1035] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:18.807849 15144 consensus_queue.cc:1035] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:18.815110 15146 mvcc.cc:205] Tried to move back new op lower bound from 7316215557344448512 to 7316215557034745856. Current Snapshot: MvccSnapshot[applied={T|T < 7316215557344448512}]
I20260808 10:37:18.818028 15147 mvcc.cc:205] Tried to move back new op lower bound from 7316215557344448512 to 7316215557034745856. Current Snapshot: MvccSnapshot[applied={T|T < 7316215557344448512}]
I20260808 10:37:18.820657 15149 mvcc.cc:205] Tried to move back new op lower bound from 7316215557344448512 to 7316215557034745856. Current Snapshot: MvccSnapshot[applied={T|T < 7316215557344448512}]
I20260808 10:37:18.826185 14614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:37:18.826375 14614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:37:18.828846 14614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Sending DeleteTablet for 3 replicas of tablet e50eaf4d26b4492983b3915e71691955
I20260808 10:37:18.829726 15047 tablet_service.cc:1514] Processing DeleteTablet for tablet e50eaf4d26b4492983b3915e71691955 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:18 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:34146
I20260808 10:37:18.829929 14748 tablet_service.cc:1514] Processing DeleteTablet for tablet e50eaf4d26b4492983b3915e71691955 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:18 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:42804
I20260808 10:37:18.830104 14905 tablet_service.cc:1514] Processing DeleteTablet for tablet e50eaf4d26b4492983b3915e71691955 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:18 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:54980
I20260808 10:37:18.830341 15156 tablet_replica.cc:330] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: stopping tablet replica
I20260808 10:37:18.830503 15156 raft_consensus.cc:2229] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:18.830680 15156 raft_consensus.cc:2258] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:18.830721 15157 tablet_replica.cc:330] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: stopping tablet replica
I20260808 10:37:18.830901 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 14656
I20260808 10:37:18.832151 15157 raft_consensus.cc:2229] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:37:18.832520 15157 raft_consensus.cc:2258] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:18.835510 15157 ts_tablet_manager.cc:1905] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:18.837999 15157 ts_tablet_manager.cc:1918] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:18.838122 15157 log.cc:1196] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/e50eaf4d26b4492983b3915e71691955
I20260808 10:37:18.838471 15157 ts_tablet_manager.cc:1939] T e50eaf4d26b4492983b3915e71691955 P 119a63ff2f7d497ba776b91b47e67def: Deleting consensus metadata
I20260808 10:37:18.839589 14597 catalog_manager.cc:4841] TS 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887): tablet e50eaf4d26b4492983b3915e71691955 (table test-table [id=470eb8c3eb014bfbbd338f3c216e6771]) successfully deleted
I20260808 10:37:18.841343 15155 tablet_replica.cc:330] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: stopping tablet replica
I20260808 10:37:18.841773 15155 raft_consensus.cc:2229] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:18.842123 15155 raft_consensus.cc:2258] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 10:37:18.842964 14598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:36921 (error 108)
I20260808 10:37:18.843498 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 14827
I20260808 10:37:18.844209 15131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:18.843803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36921 (local address 127.0.0.1:42816)
0808 10:37:18.844015 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:18.844088 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:36921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260808 10:37:18.844488 14598 catalog_manager.cc:4584] TS b9175007187148cd8c27f9344578de42 (127.30.146.129:36921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e50eaf4d26b4492983b3915e71691955: Network error: Client connection negotiation failed: client connection to 127.30.146.129:36921: connect: Connection refused (error 111)
I20260808 10:37:18.844641 15155 ts_tablet_manager.cc:1905] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:18.847548 15155 ts_tablet_manager.cc:1918] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:18.847645 15155 log.cc:1196] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/e50eaf4d26b4492983b3915e71691955
I20260808 10:37:18.848001 15155 ts_tablet_manager.cc:1939] T e50eaf4d26b4492983b3915e71691955 P a16be21fe4d44a8d9cf57d007e857379: Deleting consensus metadata
I20260808 10:37:18.849359 14597 catalog_manager.cc:4841] TS a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571): tablet e50eaf4d26b4492983b3915e71691955 (table test-table [id=470eb8c3eb014bfbbd338f3c216e6771]) successfully deleted
I20260808 10:37:18.852022 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 14979
I20260808 10:37:18.859650 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 14578
I20260808 10:37:18.868871 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33057
--webserver_interface=127.30.146.190
--webserver_port=38113
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:33057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:18.987815 15159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:18.988129 15159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:18.988214 15159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:18.992305 15159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:18.992393 15159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:18.992432 15159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:18.992488 15159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:18.992539 15159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:18.997912 15159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:33057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=38113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:18.998996 15159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:19.000036 15159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:19.009470 15164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:19.009478 15165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:19.009474 15167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:19.009591 15159 server_base.cc:1034] running on GCE node
I20260808 10:37:19.010466 15159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:19.011624 15159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:19.012953 15159 hybrid_clock.cc:648] HybridClock initialized: now 1786185439012933 us; error 29 us; skew 500 ppm
Aug 08 10:37:19 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185439, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:19.016548 15159 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:37:19.017663 15159 webserver.cc:460] Webserver started at http://127.30.146.190:38113/ using document root <none> and password file <none>
I20260808 10:37:19.018247 15159 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:19.018328 15159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:19.022842 15159 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:37:19.025414 15175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:19.026923 15159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:19.027036 15159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "c2b184d8c0674d81b9aac53d8717d8b0"
format_stamp: "Formatted at 2026-08-08 10:37:16 on dist-test-slave-9gcw"
server_key: "8ab563a642febd3aa3e0abee657c30d0"
server_key_iv: "08c23d8c8c87719a20752ab2d6836bff"
server_key_version: "encryptionkey@0"
I20260808 10:37:19.027640 15159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:19.041492 15159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:19.044694 15159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:19.044858 15159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:19.052605 15159 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:33057
I20260808 10:37:19.052634 15234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:33057 every 8 connection(s)
I20260808 10:37:19.053785 15159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:37:19.056371 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 15159
I20260808 10:37:19.056840 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:36921
--local_ip_for_outbound_sockets=127.30.146.129
--tserver_master_addrs=127.30.146.190:33057
--webserver_port=45269
--webserver_interface=127.30.146.129
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:37:19.058408 15235 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:37:19.060897 15235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Bootstrap starting.
I20260808 10:37:19.083909 15235 log.cc:824] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:19.096437 15235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:37:19.096876 15235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0: Bootstrap complete.
I20260808 10:37:19.101509 15235 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:19.101926 15235 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b184d8c0674d81b9aac53d8717d8b0, State: Initialized, Role: FOLLOWER
I20260808 10:37:19.102442 15235 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:19.102571 15235 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:37:19.102675 15235 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:37:19.102803 15235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:37:19.104967 15235 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:19.105402 15235 leader_election.cc:302] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2b184d8c0674d81b9aac53d8717d8b0; no voters:
I20260808 10:37:19.105840 15235 leader_election.cc:288] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 10:37:19.106055 15240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:37:19.106331 15240 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [term 3 LEADER]: Becoming Leader. State: Replica: c2b184d8c0674d81b9aac53d8717d8b0, State: Running, Role: LEADER
I20260808 10:37:19.106693 15240 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } }
I20260808 10:37:19.107182 15235 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:19.108018 15242 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2b184d8c0674d81b9aac53d8717d8b0. Latest consensus state: current_term: 3 leader_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } } }
I20260808 10:37:19.108179 15242 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:19.108541 15241 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b184d8c0674d81b9aac53d8717d8b0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33057 } } }
I20260808 10:37:19.108671 15241 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b184d8c0674d81b9aac53d8717d8b0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:19.109485 15249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:37:19.114061 15249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:37:19.114603 15249 catalog_manager.cc:1259] Loaded cluster ID: a7f1ea174d8d4918a092746bdb34cdd0
I20260808 10:37:19.114675 15249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 10:37:19.115567 15249 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5b26a9a9ad google::LogMessage::Fail() at ??:0
@ 0x7f5b26a9d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f5b26a9a4de google::LogMessage::Flush() at ??:0
@ 0x7f5b26a9ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5b313ddfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f5b313ded2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5b313db579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f5b3141f640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5b28cec3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5b28cecc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f5b28cee3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5b28cdd512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5b2a3216db start_thread at ??:0
@ 0x7f5b2559861f clone at ??:0
W20260808 10:37:19.216848 15238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:19.217131 15238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:19.217284 15238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:19.221719 15238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:19.221781 15238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:19.221873 15238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:37:19.227269 15238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:36921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=45269
--tserver_master_addrs=127.30.146.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:19.228358 15238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:19.229475 15238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:19.240473 15281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:19.240480 15278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:19.240794 15238 server_base.cc:1034] running on GCE node
W20260808 10:37:19.240862 15279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:19.241315 15238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:19.242213 15238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:19.243431 15238 hybrid_clock.cc:648] HybridClock initialized: now 1786185439243398 us; error 44 us; skew 500 ppm
Aug 08 10:37:19 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185439, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:19.247483 15238 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:37:19.248658 15238 webserver.cc:460] Webserver started at http://127.30.146.129:45269/ using document root <none> and password file <none>
I20260808 10:37:19.249486 15238 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:19.249607 15238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:19.255162 15238 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:37:19.258265 15289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:19.260012 15238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:37:19.260164 15238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "b9175007187148cd8c27f9344578de42"
format_stamp: "Formatted at 2026-08-08 10:37:16 on dist-test-slave-9gcw"
server_key: "ee5a3d3262304e6e058fcf5437ee1f76"
server_key_iv: "71a35e03f578d7f2c4c2b3d5d184ac1a"
server_key_version: "encryptionkey@0"
I20260808 10:37:19.260974 15238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:19.284848 15238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:19.289552 15238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:19.289806 15238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:19.290625 15238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:19.292140 15297 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:37:19.296779 15238 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:37:19.296869 15238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 10:37:19.296944 15238 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:37:19.299654 15238 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:37:19.299727 15238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260808 10:37:19.300117 15297 tablet_bootstrap.cc:492] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Bootstrap starting.
I20260808 10:37:19.311930 15238 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:36921
I20260808 10:37:19.312274 15416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:36921 every 8 connection(s)
I20260808 10:37:19.313287 15238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:37:19.316659 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 15238
I20260808 10:37:19.317284 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:37887
--local_ip_for_outbound_sockets=127.30.146.130
--tserver_master_addrs=127.30.146.190:33057
--webserver_port=35787
--webserver_interface=127.30.146.130
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:37:19.324009 15297 log.cc:824] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:19.334372 15297 tablet_bootstrap.cc:492] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:37:19.334798 15297 tablet_bootstrap.cc:492] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Bootstrap complete.
I20260808 10:37:19.336550 15297 ts_tablet_manager.cc:1397] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Time spent bootstrapping tablet: real 0.037s user 0.028s sys 0.003s
I20260808 10:37:19.339953 15297 raft_consensus.cc:348] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:19.340509 15297 raft_consensus.cc:729] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9175007187148cd8c27f9344578de42, State: Initialized, Role: FOLLOWER
I20260808 10:37:19.341032 15297 consensus_queue.cc:260] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:19.342170 15297 ts_tablet_manager.cc:1428] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
W20260808 10:37:19.367338 15419 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:37:19.315736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:37121)
0808 10:37:19.316967 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:19.317031 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:19.318493 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:19.366483 (+ 47990us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":198,"threads_started":1}
W20260808 10:37:19.368031 15417 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:33057 (0 consecutive failures): Network error: Failed to ping master at 127.30.146.190:33057: Client connection negotiation failed: client connection to 127.30.146.190:33057: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:37:19.368847 15419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:19.368550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:51991)
0808 10:37:19.368687 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:19.368768 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 10:37:19.369678 15419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:19.369461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:59867)
0808 10:37:19.369576 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:19.369623 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:37:19.369891 15417 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:37:19.449440 15420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:19.449723 15420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:19.449769 15420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:19.453944 15420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:19.454008 15420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:19.454093 15420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:37:19.459582 15420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:37887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=35787
--tserver_master_addrs=127.30.146.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:19.460721 15420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:19.461812 15420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:19.472604 15429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:19.472666 15430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:19.472734 15432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:19.472867 15420 server_base.cc:1034] running on GCE node
I20260808 10:37:19.473388 15420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:19.474100 15420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:19.475301 15420 hybrid_clock.cc:648] HybridClock initialized: now 1786185439475285 us; error 27 us; skew 500 ppm
Aug 08 10:37:19 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185439, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:19.478330 15420 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:37:19.479372 15420 webserver.cc:460] Webserver started at http://127.30.146.130:35787/ using document root <none> and password file <none>
I20260808 10:37:19.479941 15420 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:19.480036 15420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:19.484153 15420 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:37:19.486447 15439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:19.487788 15420 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:37:19.487927 15420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "119a63ff2f7d497ba776b91b47e67def"
format_stamp: "Formatted at 2026-08-08 10:37:17 on dist-test-slave-9gcw"
server_key: "aaabfe5acdccc59e34f81d66cb557da3"
server_key_iv: "4a4adf50e02a8aec1205e4da3e0bd3e3"
server_key_version: "encryptionkey@0"
I20260808 10:37:19.488507 15420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:19.508179 15420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:19.511664 15420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:19.511899 15420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:19.512549 15420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:19.513583 15420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:19.513665 15420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:19.513743 15420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:19.513798 15420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:19.523340 15420 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:37887
I20260808 10:37:19.523360 15561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:37887 every 8 connection(s)
I20260808 10:37:19.524384 15420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:37:19.526609 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 15420
I20260808 10:37:19.527105 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:38571
--local_ip_for_outbound_sockets=127.30.146.131
--tserver_master_addrs=127.30.146.190:33057
--webserver_port=40789
--webserver_interface=127.30.146.131
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:37:19.529223 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:19.526961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:50257)
0808 10:37:19.527725 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:19.528218 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":194,"threads_started":1}
W20260808 10:37:19.529713 15562 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:33057 (0 consecutive failures): Network error: Failed to ping master at 127.30.146.190:33057: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
I20260808 10:37:19.530421 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:19.530114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:48719)
0808 10:37:19.530264 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:19.530336 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 10:37:19.531225 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:19.530916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:43295)
0808 10:37:19.531062 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:19.531137 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:37:19.531524 15562 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:37:19.648772 15565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:19.649046 15565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:19.649115 15565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:19.653235 15565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:19.653297 15565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:19.653403 15565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:37:19.658846 15565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:38571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=40789
--tserver_master_addrs=127.30.146.190:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:19.659986 15565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:19.661049 15565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:19.671233 15574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:19.671268 15572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:19.671330 15571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:19.671859 15565 server_base.cc:1034] running on GCE node
I20260808 10:37:19.672266 15565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:19.672866 15565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:19.674043 15565 hybrid_clock.cc:648] HybridClock initialized: now 1786185439674007 us; error 47 us; skew 500 ppm
Aug 08 10:37:19 dist-test-slave-9gcw krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185439, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:19.676888 15565 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:37:19.677845 15565 webserver.cc:460] Webserver started at http://127.30.146.131:40789/ using document root <none> and password file <none>
I20260808 10:37:19.678412 15565 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:19.678476 15565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:19.682418 15565 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:37:19.684732 15586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:19.686133 15565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:19.686275 15565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "a16be21fe4d44a8d9cf57d007e857379"
format_stamp: "Formatted at 2026-08-08 10:37:17 on dist-test-slave-9gcw"
server_key: "96dc1708d8447461e6572ee8e7b70bd0"
server_key_iv: "c200ede1b7223c2feff64d9fc8907295"
server_key_version: "encryptionkey@0"
I20260808 10:37:19.686828 15565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:19.709769 15565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:19.712906 15565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:19.713121 15565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:19.713847 15565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:19.714716 15565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:19.714777 15565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:19.714854 15565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:19.714898 15565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:19.725184 15565 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:38571
I20260808 10:37:19.725208 15706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:38571 every 8 connection(s)
I20260808 10:37:19.726338 15565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:37:19.730537 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:19.728718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:39733)
0808 10:37:19.729264 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:19.729741 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":121,"threads_started":1}
W20260808 10:37:19.731187 15707 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:33057 (0 consecutive failures): Network error: Failed to ping master at 127.30.146.190:33057: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
I20260808 10:37:19.731781 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:19.731535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:57435)
0808 10:37:19.731663 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:19.731721 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:37:19.732506 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:19.732251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:51923)
0808 10:37:19.732394 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:19.732447 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 10:37:19.732735 15707 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:37:19.736308 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 15565
I20260808 10:37:20.371065 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:20.370560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:43133)
0808 10:37:20.370868 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:20.370939 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":95,"threads_started":1}
I20260808 10:37:20.532529 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:20.532074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:39559)
0808 10:37:20.532354 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:20.532413 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":73,"threads_started":1}
I20260808 10:37:20.733830 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:20.733387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:40567)
0808 10:37:20.733652 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:20.733721 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
I20260808 10:37:20.858281 15716 raft_consensus.cc:482] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:37:20.858649 15716 raft_consensus.cc:504] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:20.859941 15716 leader_election.cc:288] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571), 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887)
Aug 08 10:37:20 dist-test-slave-9gcw krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185439, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.131@KRBTEST.COM
Aug 08 10:37:20 dist-test-slave-9gcw krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185439, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.130@KRBTEST.COM
I20260808 10:37:20.885823 15719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:20.860891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:51491 (local address 127.30.146.130:37887)
0808 10:37:20.861348 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:20.861376 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:20.861422 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:37:20.862362 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:20.862370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:20.862836 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:20.863181 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:20.865231 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:20.866859 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:20.869024 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:20.869866 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:20.870041 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:20.880380 (+ 10339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:20.880386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:20.880398 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:20.880438 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:20.882499 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:20.883508 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:20.883512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:20.883516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:20.883623 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:20.883927 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:20.883931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:20.883932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:20.884350 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:20.884480 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:20.885315 (+ 835us) server_negotiation.cc:299] Negotiation successful
0808 10:37:20.885622 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":78,"threads_started":1}
I20260808 10:37:20.885823 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:20.860433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:40585 (local address 127.30.146.131:38571)
0808 10:37:20.861348 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:20.861376 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:20.861422 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:37:20.862362 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:20.862370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:20.862854 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:20.863181 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:20.865253 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:20.866873 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:20.869022 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:20.869808 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:20.870242 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:20.880251 (+ 10009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:20.880263 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:20.880284 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:20.880318 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:20.882476 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:20.883256 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:20.883264 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:20.883268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:20.883392 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:20.883758 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:20.883761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:20.883762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:20.884348 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:20.884480 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:20.885315 (+ 835us) server_negotiation.cc:299] Negotiation successful
0808 10:37:20.885622 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":194,"threads_started":1}
I20260808 10:37:20.885823 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:20.860205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:38571 (local address 127.30.146.129:40585)
0808 10:37:20.860908 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:20.860927 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:20.861080 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:20.863314 (+ 2234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:20.863320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:20.863996 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:20.864947 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:20.864970 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:20.867060 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:20.867064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:20.868831 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:20.868842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:20.869970 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:20.869974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:20.870310 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:20.877533 (+ 7223us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:20.877565 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:20.879892 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:20.882622 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:20.882631 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:20.882644 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:20.883116 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:20.883493 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:20.883496 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:20.883497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:20.883635 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:20.884489 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:20.884496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:20.884901 (+ 405us) client_negotiation.cc:769] Sending connection context
0808 10:37:20.885165 (+ 264us) client_negotiation.cc:240] Negotiation successful
0808 10:37:20.885647 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":745,"spinlock_wait_cycles":173184}
I20260808 10:37:20.885823 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:20.860531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:37887 (local address 127.30.146.129:51491)
0808 10:37:20.860908 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:20.860927 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:20.861072 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:20.863314 (+ 2242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:20.863320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:20.863996 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:20.864947 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:20.864970 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:20.867060 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:20.867064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:20.868831 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:20.868842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:20.869981 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:20.869984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:20.870313 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:20.877533 (+ 7220us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:20.877565 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:20.880137 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:20.882639 (+ 2502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:20.882641 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:20.882646 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:20.883116 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:20.883729 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:20.883735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:20.883739 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:20.883829 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:20.884489 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:20.884496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:20.884901 (+ 405us) client_negotiation.cc:769] Sending connection context
0808 10:37:20.885165 (+ 264us) client_negotiation.cc:240] Negotiation successful
0808 10:37:20.885594 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":494,"spinlock_wait_cycles":40832,"thread_start_us":84,"threads_started":1}
W20260808 10:37:20.888692 15292 leader_election.cc:341] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571): Not found: Tablet not found: e50eaf4d26b4492983b3915e71691955
W20260808 10:37:20.888973 15292 leader_election.cc:341] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887): Not found: Tablet not found: e50eaf4d26b4492983b3915e71691955
I20260808 10:37:20.889056 15292 leader_election.cc:302] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9175007187148cd8c27f9344578de42; no voters: 119a63ff2f7d497ba776b91b47e67def, a16be21fe4d44a8d9cf57d007e857379
I20260808 10:37:20.889377 15716 raft_consensus.cc:2726] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:37:21.372219 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:21.371912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:52627)
0808 10:37:21.372051 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:21.372121 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 10:37:21.534041 15720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:21.533550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:33227)
0808 10:37:21.533846 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:21.533906 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":108,"threads_started":1}
I20260808 10:37:21.735067 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:21.734551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:49457)
0808 10:37:21.734875 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:21.734963 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":141,"threads_started":1}
I20260808 10:37:22.373852 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:22.373286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:51263)
0808 10:37:22.373634 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:22.373715 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":91,"threads_started":1}
I20260808 10:37:22.417659 15723 raft_consensus.cc:482] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:37:22.417835 15723 raft_consensus.cc:504] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:22.418387 15723 leader_election.cc:288] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571), 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887)
W20260808 10:37:22.419301 15292 leader_election.cc:341] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571): Not found: Tablet not found: e50eaf4d26b4492983b3915e71691955
W20260808 10:37:22.419448 15292 leader_election.cc:341] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887): Not found: Tablet not found: e50eaf4d26b4492983b3915e71691955
I20260808 10:37:22.419508 15292 leader_election.cc:302] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9175007187148cd8c27f9344578de42; no voters: 119a63ff2f7d497ba776b91b47e67def, a16be21fe4d44a8d9cf57d007e857379
I20260808 10:37:22.419688 15723 raft_consensus.cc:2726] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:37:22.535386 15724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:22.534925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:49227)
0808 10:37:22.535203 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:22.535269 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":82,"threads_started":1}
I20260808 10:37:22.736500 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:22.736033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:40291)
0808 10:37:22.736343 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:22.736407 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":85,"threads_started":1}
I20260808 10:37:23.375379 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:23.374834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:44689)
0808 10:37:23.375179 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:23.375250 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":95,"threads_started":1}
I20260808 10:37:23.536804 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:23.536360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:59435)
0808 10:37:23.536643 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:23.536706 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":83,"threads_started":1}
I20260808 10:37:23.737957 15728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:23.737433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:34909)
0808 10:37:23.737744 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:23.737841 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":97,"threads_started":1}
I20260808 10:37:24.377233 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:24.376515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:45809)
0808 10:37:24.376989 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:24.377053 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":177,"threads_started":1}
I20260808 10:37:24.538302 15730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:24.537775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:55613)
0808 10:37:24.538123 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:24.538195 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":98,"threads_started":1}
I20260808 10:37:24.739495 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:24.738977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:47003)
0808 10:37:24.739321 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:24.739396 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":139,"threads_started":1}
I20260808 10:37:25.247641 15732 raft_consensus.cc:482] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:37:25.247836 15732 raft_consensus.cc:504] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:25.248286 15732 leader_election.cc:288] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571), 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887)
W20260808 10:37:25.249121 15292 leader_election.cc:341] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571): Not found: Tablet not found: e50eaf4d26b4492983b3915e71691955
W20260808 10:37:25.249358 15292 leader_election.cc:341] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887): Not found: Tablet not found: e50eaf4d26b4492983b3915e71691955
I20260808 10:37:25.249432 15292 leader_election.cc:302] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9175007187148cd8c27f9344578de42; no voters: 119a63ff2f7d497ba776b91b47e67def, a16be21fe4d44a8d9cf57d007e857379
I20260808 10:37:25.249588 15732 raft_consensus.cc:2726] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:37:25.378861 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:25.378346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:58607)
0808 10:37:25.378656 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:25.378755 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":85,"threads_started":1}
I20260808 10:37:25.539709 15735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:25.539211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:57271)
0808 10:37:25.539482 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:25.539539 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":94,"threads_started":1}
I20260808 10:37:25.740976 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:25.740439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:43539)
0808 10:37:25.740803 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:25.740870 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":80,"threads_started":1}
I20260808 10:37:26.380254 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:26.379765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:43933)
0808 10:37:26.380088 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:26.380152 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":93,"threads_started":1}
I20260808 10:37:26.541128 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:26.540519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:34805)
0808 10:37:26.540912 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:26.540994 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":75,"threads_started":1}
I20260808 10:37:26.742496 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:26.741902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:37345)
0808 10:37:26.742308 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:26.742382 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":82,"threads_started":1}
I20260808 10:37:27.381754 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:27.381262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:37785)
0808 10:37:27.381563 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:27.381622 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":96,"threads_started":1}
I20260808 10:37:27.542631 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:27.542143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:43441)
0808 10:37:27.542466 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:27.542530 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":103,"threads_started":1}
I20260808 10:37:27.743829 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:27.743376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:46861)
0808 10:37:27.743660 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:27.743726 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":82,"threads_started":1}
I20260808 10:37:28.383175 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:28.382689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:55207)
0808 10:37:28.382983 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:28.383044 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":87,"threads_started":1}
I20260808 10:37:28.543982 15744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:28.543519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:53971)
0808 10:37:28.543808 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:28.543876 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
I20260808 10:37:28.745376 15745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:28.744767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:37359)
0808 10:37:28.745049 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:28.745108 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":96,"threads_started":1}
I20260808 10:37:28.963465 15746 raft_consensus.cc:482] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:37:28.963704 15746 raft_consensus.cc:504] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a16be21fe4d44a8d9cf57d007e857379" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38571 } } peers { permanent_uuid: "119a63ff2f7d497ba776b91b47e67def" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 37887 } } peers { permanent_uuid: "b9175007187148cd8c27f9344578de42" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36921 } }
I20260808 10:37:28.964155 15746 leader_election.cc:288] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571), 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887)
W20260808 10:37:28.965121 15292 leader_election.cc:341] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 119a63ff2f7d497ba776b91b47e67def (127.30.146.130:37887): Not found: Tablet not found: e50eaf4d26b4492983b3915e71691955
W20260808 10:37:28.965358 15292 leader_election.cc:341] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a16be21fe4d44a8d9cf57d007e857379 (127.30.146.131:38571): Not found: Tablet not found: e50eaf4d26b4492983b3915e71691955
I20260808 10:37:28.965451 15292 leader_election.cc:302] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9175007187148cd8c27f9344578de42; no voters: 119a63ff2f7d497ba776b91b47e67def, a16be21fe4d44a8d9cf57d007e857379
I20260808 10:37:28.965698 15746 raft_consensus.cc:2726] T e50eaf4d26b4492983b3915e71691955 P b9175007187148cd8c27f9344578de42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:37:29.384826 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:29.384364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:34345)
0808 10:37:29.384659 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:29.384726 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":94,"threads_started":1}
I20260808 10:37:29.545521 15749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:29.545030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:49841)
0808 10:37:29.545370 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:29.545425 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":161,"threads_started":1}
I20260808 10:37:29.746774 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:29.746287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:44677)
0808 10:37:29.746596 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:29.746656 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":88,"threads_started":1}
I20260808 10:37:30.386226 15751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:30.385728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:40043)
0808 10:37:30.386034 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:30.386098 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":93,"threads_started":1}
I20260808 10:37:30.546959 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:30.546409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:51005)
0808 10:37:30.546745 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:30.546826 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":100,"threads_started":1}
I20260808 10:37:30.748075 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:30.747598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:43625)
0808 10:37:30.747901 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:30.747965 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":89,"threads_started":1}
I20260808 10:37:31.387712 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:31.387174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:52903)
0808 10:37:31.387499 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:31.387584 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":89,"threads_started":1}
I20260808 10:37:31.548338 15755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:31.547878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:40507)
0808 10:37:31.548178 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:31.548237 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":95,"threads_started":1}
I20260808 10:37:31.749699 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:31.749099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:42897)
0808 10:37:31.749459 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:31.749525 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":160,"threads_started":1}
I20260808 10:37:32.389400 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:32.388796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:45357)
0808 10:37:32.389202 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:32.389285 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":102,"threads_started":1}
I20260808 10:37:32.549790 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:32.549332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:40483)
0808 10:37:32.549598 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:32.549670 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":82,"threads_started":1}
I20260808 10:37:32.751147 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:32.750642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:56117)
0808 10:37:32.750975 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:32.751041 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":89,"threads_started":1}
I20260808 10:37:33.390931 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:33.390344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:40035)
0808 10:37:33.390746 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:33.390826 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":82,"threads_started":1}
I20260808 10:37:33.551317 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:33.550821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:41515)
0808 10:37:33.551141 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:33.551209 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":101,"threads_started":1}
I20260808 10:37:33.752858 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:33.752146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:34393)
0808 10:37:33.752586 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:33.752730 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":120,"threads_started":1}
I20260808 10:37:34.392748 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:34.392288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.129:38429)
0808 10:37:34.392582 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:34.392646 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":102,"threads_started":1}
I20260808 10:37:34.552923 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:34.552412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.130:46637)
0808 10:37:34.552748 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:34.552818 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":81,"threads_started":1}
I20260808 10:37:34.737389 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 15238
I20260808 10:37:34.747233 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 15420
I20260808 10:37:34.754741 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:34.754178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33057 (local address 127.30.146.131:47849)
0808 10:37:34.754509 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:34.754577 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":98,"threads_started":1}
I20260808 10:37:34.756559 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 15565
2026-08-08T10:37:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19680 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:37:34 dist-test-slave-9gcw krb5kdc[15769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:37:34 dist-test-slave-9gcw krb5kdc[15769](info): set up 2 sockets
Aug 08 10:37:34 dist-test-slave-9gcw krb5kdc[15769](info): commencing operation
krb5kdc: starting...
W20260808 10:37:35.881625 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:37:35 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:37:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:37:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:37:36.024380 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33317
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:33317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:36.151727 15783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:36.152078 15783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:36.152161 15783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:36.157662 15783 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:37:36.157747 15783 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:37:36.157775 15783 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:37:36.157802 15783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:36.157845 15783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:36.157874 15783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:36.157904 15783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:36.157940 15783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:36.164458 15783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:33317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:36.165727 15783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:36.166921 15783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:37:36.177567 15783 server_base.cc:1034] running on GCE node
W20260808 10:37:36.177506 15789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:36.177513 15791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:36.177531 15788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:36.178550 15783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:36.179816 15783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:36.181026 15783 hybrid_clock.cc:648] HybridClock initialized: now 1786185456181002 us; error 44 us; skew 500 ppm
Aug 08 10:37:36 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:36.184176 15783 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:37:36.185307 15783 webserver.cc:460] Webserver started at http://127.30.146.190:42431/ using document root <none> and password file <none>
I20260808 10:37:36.185962 15783 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:36.186069 15783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:36.186342 15783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:36.188740 15783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "b6b79a18f3894a5486b416a4d8ea5641"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "f9a38043a24c097ad6bcbf4d1e800db3"
server_key_iv: "f8bca7e6c6c667eb849b3c4f9deaede9"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.189440 15783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "b6b79a18f3894a5486b416a4d8ea5641"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "f9a38043a24c097ad6bcbf4d1e800db3"
server_key_iv: "f8bca7e6c6c667eb849b3c4f9deaede9"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.193728 15783 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 10:37:36.196405 15800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:36.197849 15783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:36.198004 15783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "b6b79a18f3894a5486b416a4d8ea5641"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "f9a38043a24c097ad6bcbf4d1e800db3"
server_key_iv: "f8bca7e6c6c667eb849b3c4f9deaede9"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.198125 15783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:36.230496 15783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:36.233983 15783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:36.234282 15783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:36.242204 15783 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:33317
I20260808 10:37:36.242491 15855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:33317 every 8 connection(s)
I20260808 10:37:36.243357 15783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:37:36.246459 15856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:36.252444 15856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Bootstrap starting.
I20260808 10:37:36.252864 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 15783
I20260808 10:37:36.253028 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:37:36.253414 31306 external_mini_cluster.cc:1428] Setting key d389aa6988662350fc96956734aa2799
I20260808 10:37:36.255451 15856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:36.256287 15856 log.cc:824] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:36.258724 15856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: No bootstrap required, opened a new log
Aug 08 10:37:36 dist-test-slave-9gcw krb5kdc[15769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:36.263784 15856 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:37:36.264353 15856 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:36.264460 15856 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b79a18f3894a5486b416a4d8ea5641, State: Initialized, Role: FOLLOWER
I20260808 10:37:36.264973 15856 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:37:36.265182 15856 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:37:36.265280 15856 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:37:36.265398 15856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:36.266610 15856 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:37:36.266968 15856 leader_election.cc:302] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6b79a18f3894a5486b416a4d8ea5641; no voters:
I20260808 10:37:36.267412 15856 leader_election.cc:288] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:37:36.267525 15862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:37:36.267908 15862 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 1 LEADER]: Becoming Leader. State: Replica: b6b79a18f3894a5486b416a4d8ea5641, State: Running, Role: LEADER
I20260808 10:37:36.268318 15862 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:37:36.269009 15856 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:36.270533 15864 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6b79a18f3894a5486b416a4d8ea5641. Latest consensus state: current_term: 1 leader_uuid: "b6b79a18f3894a5486b416a4d8ea5641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } } }
I20260808 10:37:36.270561 15863 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6b79a18f3894a5486b416a4d8ea5641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } } }
I20260808 10:37:36.270756 15864 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:36.270774 15863 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:36.271085 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:36.254908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48884 (local address 127.30.146.190:33317)
0808 10:37:36.255366 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:36.255384 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:36.255433 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:37:36.256184 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:36.256201 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:36.256592 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:36.256984 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:36.257966 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:36.259140 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:36.260477 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:36.261112 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:36.261333 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:36.264815 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:36.264823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:36.264852 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:36.264881 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:36.267012 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:36.267657 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:36.267666 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:36.267674 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:36.267844 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:36.268194 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:36.268198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:36.268200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:36.268765 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:36.269006 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:36.269505 (+ 499us) server_negotiation.cc:299] Negotiation successful
0808 10:37:36.269855 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":136,"threads_started":1}
W20260808 10:37:36.275246 15879 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:37:36.275372 15879 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:37:36.275480 15876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:37:36.276469 15876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:37:36.282655 15876 catalog_manager.cc:1347] Generated new cluster ID: f2be7bf1d1f64167bf0235ffb3f2f36f
I20260808 10:37:36.282735 15876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:37:36.296825 15876 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:37:36.297959 15876 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:37:36.309015 15876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Generated new TSK 0
I20260808 10:37:36.309816 15876 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:37:36.348935 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33317
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:36.472697 15886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:36.472970 15886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:36.473014 15886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:36.477319 15886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:36.477386 15886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:36.477448 15886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:37:36.483016 15886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:36.484165 15886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:36.485277 15886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:37:36.495958 15886 server_base.cc:1034] running on GCE node
W20260808 10:37:36.495983 15892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:36.495906 15894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:36.495908 15891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:36.496634 15886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:36.497403 15886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:36.498593 15886 hybrid_clock.cc:648] HybridClock initialized: now 1786185456498553 us; error 51 us; skew 500 ppm
Aug 08 10:37:36 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:36.502018 15886 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:37:36.503106 15886 webserver.cc:460] Webserver started at http://127.30.146.129:46075/ using document root <none> and password file <none>
I20260808 10:37:36.503741 15886 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:36.503810 15886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:36.504067 15886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:36.506412 15886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "9310e650d22e4d3889b6186f9f3a2247"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "d972e00e2f48a0172d6fa810d64f651c"
server_key_iv: "f7b57aa37a074af82046d4b6fecca154"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.507030 15886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "9310e650d22e4d3889b6186f9f3a2247"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "d972e00e2f48a0172d6fa810d64f651c"
server_key_iv: "f7b57aa37a074af82046d4b6fecca154"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.511297 15886 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:37:36.514066 15901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:36.515414 15886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:36.515550 15886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "9310e650d22e4d3889b6186f9f3a2247"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "d972e00e2f48a0172d6fa810d64f651c"
server_key_iv: "f7b57aa37a074af82046d4b6fecca154"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.515673 15886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:36.538585 15886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:36.541956 15886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:36.542181 15886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:36.542877 15886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:36.543807 15886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:36.543886 15886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:36.543970 15886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:36.544019 15886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:36.554625 15886 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:46423
I20260808 10:37:36.554721 16019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:46423 every 8 connection(s)
I20260808 10:37:36.555707 15886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:37:36.557394 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 15886
I20260808 10:37:36.557556 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:37:36.557978 31306 external_mini_cluster.cc:1428] Setting key f358ca2405628a3d0745823afc654f36
Aug 08 10:37:36 dist-test-slave-9gcw krb5kdc[15769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:37:36.572988 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:36.557962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:52377 (local address 127.30.146.190:33317)
0808 10:37:36.558132 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:36.558137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:36.559429 (+ 1292us) server_negotiation.cc:407] Connection header received
0808 10:37:36.560398 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:36.560402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:36.560460 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:36.560547 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:36.562054 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:36.562627 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:36.564610 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:36.564865 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:36.565231 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:36.568615 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:36.568625 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:36.568626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:36.568652 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:36.570687 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:36.571296 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:36.571299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:36.571301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:36.571392 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:36.571741 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:36.571744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:36.571745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:36.572020 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:36.572127 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:36.572671 (+ 544us) server_negotiation.cc:299] Negotiation successful
0808 10:37:36.572817 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:37:36.574227 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:36.558360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:52377)
0808 10:37:36.559240 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:36.559301 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:36.560177 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:36.560736 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:36.560744 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:36.561197 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:36.561853 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:36.561878 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:36.562789 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:36.562795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:36.564449 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:36.564466 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:36.565069 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:36.565076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:36.565475 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:36.566291 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:36.566320 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:36.568434 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:36.570865 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:36.570871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:36.570889 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:36.571180 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:36.571486 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:36.571489 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:36.571490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:36.571628 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:36.572126 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:36.572132 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:36.572396 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 10:37:36.572623 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 10:37:36.573176 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":140,"threads_started":1}
I20260808 10:37:36.575732 16020 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33317
I20260808 10:37:36.576017 16020 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:36.576591 16020 heartbeater.cc:511] Master 127.30.146.190:33317 requested a full tablet report, sending...
I20260808 10:37:36.578331 15818 ts_manager.cc:194] Registered new tserver with Master: 9310e650d22e4d3889b6186f9f3a2247 (127.30.146.129:46423)
I20260808 10:37:36.579545 15818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:52377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:37:36.596681 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33317
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:36.716666 16029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:36.716926 16029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:36.716970 16029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:36.721174 16029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:36.721238 16029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:36.721297 16029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:37:36.726547 16029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:36.727532 16029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:36.728492 16029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:36.739141 16037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:36.739244 16029 server_base.cc:1034] running on GCE node
W20260808 10:37:36.739126 16034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:36.739126 16035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:36.739845 16029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:36.740491 16029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:36.741684 16029 hybrid_clock.cc:648] HybridClock initialized: now 1786185456741655 us; error 40 us; skew 500 ppm
Aug 08 10:37:36 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:36.744777 16029 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:37:36.745765 16029 webserver.cc:460] Webserver started at http://127.30.146.130:45757/ using document root <none> and password file <none>
I20260808 10:37:36.746347 16029 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:36.746443 16029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:36.746692 16029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:36.748759 16029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "e02f043225a1467c98355320ed4ce356"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "f5673d70d7ffab605ec5b1ad92480006"
server_key_iv: "2500cc6151dee5de61e24cf751fcb4bd"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.749415 16029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "e02f043225a1467c98355320ed4ce356"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "f5673d70d7ffab605ec5b1ad92480006"
server_key_iv: "2500cc6151dee5de61e24cf751fcb4bd"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.753602 16029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:37:36.756373 16045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:36.757882 16029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:36.758034 16029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "e02f043225a1467c98355320ed4ce356"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "f5673d70d7ffab605ec5b1ad92480006"
server_key_iv: "2500cc6151dee5de61e24cf751fcb4bd"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.758165 16029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:36.777339 16029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:36.780748 16029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:36.780959 16029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:36.781723 16029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:36.782624 16029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:36.782699 16029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:36.782765 16029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:36.782814 16029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:36.792665 16029 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:38867
I20260808 10:37:36.792708 16170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:38867 every 8 connection(s)
I20260808 10:37:36.793696 16029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:37:36.795027 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 16029
I20260808 10:37:36.795186 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:37:36.795532 31306 external_mini_cluster.cc:1428] Setting key df4d175afdd5814a74ef9b87b8622a2c
Aug 08 10:37:36 dist-test-slave-9gcw krb5kdc[15769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:37:36.809576 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:36.795789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:45559 (local address 127.30.146.190:33317)
0808 10:37:36.795956 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:36.795961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:36.797229 (+ 1268us) server_negotiation.cc:407] Connection header received
0808 10:37:36.798171 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:36.798177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:36.798239 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:36.798344 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:36.799601 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:36.800469 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:36.801761 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:36.802244 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:36.802371 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:36.805330 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:36.805335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:36.805341 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:36.805374 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:36.807018 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:36.807594 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:36.807597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:36.807598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:36.807677 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:36.808040 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:36.808043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:36.808044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:36.808336 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:36.808495 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:36.809239 (+ 744us) server_negotiation.cc:299] Negotiation successful
0808 10:37:36.809381 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
I20260808 10:37:36.810527 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:36.796289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:45559)
0808 10:37:36.797044 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:36.797098 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:36.797939 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:36.798505 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:36.798512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:36.798924 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:36.799445 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:36.799459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:36.800618 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:36.800620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:36.801636 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:36.801646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:36.802488 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:36.802491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:36.802637 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:36.803338 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:36.803355 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:36.805096 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:36.807168 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:36.807211 (+ 43us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:36.807223 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:36.807479 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:36.807765 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:36.807767 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:36.807768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:36.807906 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:36.808495 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:36.808504 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:36.808931 (+ 427us) client_negotiation.cc:769] Sending connection context
0808 10:37:36.809139 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 10:37:36.809535 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":144,"threads_started":1}
I20260808 10:37:36.811862 16171 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33317
I20260808 10:37:36.812150 16171 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:36.812686 16171 heartbeater.cc:511] Master 127.30.146.190:33317 requested a full tablet report, sending...
I20260808 10:37:36.813886 15818 ts_manager.cc:194] Registered new tserver with Master: e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867)
I20260808 10:37:36.814524 15818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:45559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:37:36.828944 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33317
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:36.949309 16176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:36.949571 16176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:36.949615 16176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:36.953696 16176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:36.953763 16176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:36.953822 16176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:37:36.959259 16176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:36.960269 16176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:36.961318 16176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:36.972082 16182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:36.972275 16176 server_base.cc:1034] running on GCE node
W20260808 10:37:36.972095 16181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:36.972086 16184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:36.972906 16176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:36.973678 16176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:36.974887 16176 hybrid_clock.cc:648] HybridClock initialized: now 1786185456974869 us; error 30 us; skew 500 ppm
Aug 08 10:37:36 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:36.977978 16176 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:37:36.978888 16176 webserver.cc:460] Webserver started at http://127.30.146.131:42747/ using document root <none> and password file <none>
I20260808 10:37:36.979468 16176 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:36.979566 16176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:36.979826 16176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:37:36.982053 16176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "662378f8eb92444f9ed9d9a6b87638b4"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "4bc2bfb82cdfa88a194f284d6dca2dd1"
server_key_iv: "2e530e075049d5441e5993d282477e9e"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.982710 16176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "662378f8eb92444f9ed9d9a6b87638b4"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "4bc2bfb82cdfa88a194f284d6dca2dd1"
server_key_iv: "2e530e075049d5441e5993d282477e9e"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.986784 16176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:37:36.989691 16191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:36.991143 16176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:36.991302 16176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "662378f8eb92444f9ed9d9a6b87638b4"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "4bc2bfb82cdfa88a194f284d6dca2dd1"
server_key_iv: "2e530e075049d5441e5993d282477e9e"
server_key_version: "encryptionkey@0"
I20260808 10:37:36.991432 16176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:37.006697 16176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:37.009846 16176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:37.010072 16176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:37.010727 16176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:37.011610 16176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:37.011687 16176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:37.011762 16176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:37.011816 16176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:37.021322 16176 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:39111
I20260808 10:37:37.021359 16313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:39111 every 8 connection(s)
I20260808 10:37:37.022238 16176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:37:37.025009 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 16176
I20260808 10:37:37.025172 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:37:37.025516 31306 external_mini_cluster.cc:1428] Setting key 61e8959206f582a03365026747e007fb
Aug 08 10:37:37 dist-test-slave-9gcw krb5kdc[15769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185456, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:37.036878 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.024115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:34237 (local address 127.30.146.190:33317)
0808 10:37:37.024266 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:37.024269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:37.025509 (+ 1240us) server_negotiation.cc:407] Connection header received
0808 10:37:37.026819 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:37.026822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:37.026886 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:37.026977 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:37.028347 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.028936 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.030179 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.030402 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.030490 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.033240 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:37.033243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:37.033245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:37.033270 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:37.034968 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:37.035473 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:37.035475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:37.035477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:37.035553 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:37.035877 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:37.035879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:37.035881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:37.036066 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:37.036142 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:37.036631 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 10:37:37.036744 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:37:37.037832 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.024496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:34237)
0808 10:37:37.025328 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:37.025392 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:37.026445 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:37.027195 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:37.027202 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:37.027560 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:37.028201 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.028215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.029055 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.029057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.030035 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.030069 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.030542 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.030544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.030690 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.031260 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:37.031277 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:37.032989 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:37.035077 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:37.035082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:37.035095 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:37.035383 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:37.035632 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:37.035634 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:37.035636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:37.035775 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:37.036157 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:37.036162 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:37.036420 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 10:37:37.036583 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 10:37:37.036875 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":126,"threads_started":1}
I20260808 10:37:37.039125 16314 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33317
I20260808 10:37:37.039384 16314 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:37.039966 16314 heartbeater.cc:511] Master 127.30.146.190:33317 requested a full tablet report, sending...
I20260808 10:37:37.040972 15818 ts_manager.cc:194] Registered new tserver with Master: 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111)
I20260808 10:37:37.041564 15818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:34237
I20260808 10:37:37.052014 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:37:37.062134 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.054187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.30.146.190:33317)
0808 10:37:37.054317 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:37.054321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:37.054500 (+ 179us) server_negotiation.cc:407] Connection header received
0808 10:37:37.054601 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:37.054603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:37.054642 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:37.054757 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:37.055464 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.056005 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.056847 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.057106 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.057197 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.058815 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:37.058818 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:37.058819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:37.058841 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:37.060319 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:37.060839 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:37.060842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:37.060843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:37.060922 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:37.061294 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:37.061296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:37.061297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:37.061587 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:37.061697 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:37.061906 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 10:37:37.062007 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:37:37.065254 15818 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:37:37.068300 15818 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:37:37.083437 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.077286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:46423 (local address 127.0.0.1:44498)
0808 10:37:37.077998 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:37.078052 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:37.078214 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:37.078996 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:37.079002 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:37.079026 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:37.079180 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.079189 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.080352 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.080357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.081666 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.081686 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.082800 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.082803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.082932 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.082977 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 10:37:37.083088 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:37:37.083245 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":2,"thread_start_us":108,"threads_started":1}
I20260808 10:37:37.083626 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.077287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.30.146.131:39111)
0808 10:37:37.078502 (+ 1215us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:37.078513 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:37.078533 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:37:37.078613 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:37.078618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:37.078834 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:37.079013 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:37.079465 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.080645 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.082287 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.083082 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.083220 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.083336 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:37.083394 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 10:37:37.083490 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1117,"thread_start_us":154,"threads_started":1}
I20260808 10:37:37.083842 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.077897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59470 (local address 127.30.146.130:38867)
0808 10:37:37.078315 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:37.078329 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:37.078683 (+ 354us) server_negotiation.cc:407] Connection header received
0808 10:37:37.078970 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:37.078976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:37.079174 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:37.079296 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:37.079700 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.080804 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.082213 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.082827 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.082948 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.083072 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:37.083499 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 10:37:37.083593 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":152,"threads_started":1}
I20260808 10:37:37.083433 16325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.077514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:38867 (local address 127.0.0.1:59470)
0808 10:37:37.078578 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:37.078591 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:37.078757 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:37.079487 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:37.079489 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:37.079506 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:37.079579 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.079585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.080940 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.080947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.082061 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.082071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.083005 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.083009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.083095 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.083129 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 10:37:37.083198 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:37:37.083267 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"thread_start_us":93,"threads_started":1}
I20260808 10:37:37.083630 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.077069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:39111 (local address 127.0.0.1:46002)
0808 10:37:37.077999 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:37.078056 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:37.078191 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:37.079188 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:37.079190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:37.079205 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:37.079322 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.079329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.080799 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.080802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.082140 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.082149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.083200 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.083203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.083284 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.083305 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:37:37.083370 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:37:37.083424 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":228,"spinlock_wait_cycles":1920,"thread_start_us":119,"threads_started":1}
I20260808 10:37:37.085058 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.077878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.30.146.129:46423)
0808 10:37:37.078502 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:37.078510 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:37.078533 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:37:37.078612 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:37.078618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:37.078843 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:37.079013 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:37.079317 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.080233 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.081818 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.082639 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.082776 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.082919 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:37.083904 (+ 985us) server_negotiation.cc:299] Negotiation successful
0808 10:37:37.084020 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":93,"threads_started":1}
I20260808 10:37:37.086984 16099 tablet_service.cc:1467] Processing CreateTablet for tablet 6729c3d1f46044e59a6227c447ce6778 (DEFAULT_TABLE table=test-table [id=58c6e4aff8474d09acfd995045607fd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:37.087096 16242 tablet_service.cc:1467] Processing CreateTablet for tablet 6729c3d1f46044e59a6227c447ce6778 (DEFAULT_TABLE table=test-table [id=58c6e4aff8474d09acfd995045607fd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:37.087518 15951 tablet_service.cc:1467] Processing CreateTablet for tablet 6729c3d1f46044e59a6227c447ce6778 (DEFAULT_TABLE table=test-table [id=58c6e4aff8474d09acfd995045607fd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:37.088095 16099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6729c3d1f46044e59a6227c447ce6778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:37.088163 16242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6729c3d1f46044e59a6227c447ce6778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:37.088464 15951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6729c3d1f46044e59a6227c447ce6778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:37.094982 16331 tablet_bootstrap.cc:492] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: Bootstrap starting.
I20260808 10:37:37.096433 16332 tablet_bootstrap.cc:492] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Bootstrap starting.
I20260808 10:37:37.097529 16333 tablet_bootstrap.cc:492] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: Bootstrap starting.
I20260808 10:37:37.098075 16331 tablet_bootstrap.cc:654] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:37.098615 16332 tablet_bootstrap.cc:654] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:37.099500 16332 log.cc:824] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:37.099503 16331 log.cc:824] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:37.100106 16333 tablet_bootstrap.cc:654] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:37.100980 16333 log.cc:824] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:37.101908 16331 tablet_bootstrap.cc:492] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: No bootstrap required, opened a new log
I20260808 10:37:37.102129 16331 ts_tablet_manager.cc:1397] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:37:37.102674 16332 tablet_bootstrap.cc:492] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: No bootstrap required, opened a new log
I20260808 10:37:37.102897 16332 ts_tablet_manager.cc:1397] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260808 10:37:37.103077 16333 tablet_bootstrap.cc:492] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: No bootstrap required, opened a new log
I20260808 10:37:37.103266 16333 ts_tablet_manager.cc:1397] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:37:37.106299 16331 raft_consensus.cc:348] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:37.106600 16331 raft_consensus.cc:374] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:37.106654 16331 raft_consensus.cc:729] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02f043225a1467c98355320ed4ce356, State: Initialized, Role: FOLLOWER
I20260808 10:37:37.107071 16333 raft_consensus.cc:348] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:37.107148 16331 consensus_queue.cc:260] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:37.107335 16333 raft_consensus.cc:374] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:37.107188 16332 raft_consensus.cc:348] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:37.107390 16333 raft_consensus.cc:729] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 662378f8eb92444f9ed9d9a6b87638b4, State: Initialized, Role: FOLLOWER
I20260808 10:37:37.107422 16332 raft_consensus.cc:374] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:37.107476 16332 raft_consensus.cc:729] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9310e650d22e4d3889b6186f9f3a2247, State: Initialized, Role: FOLLOWER
I20260808 10:37:37.107887 16333 consensus_queue.cc:260] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:37.107944 16332 consensus_queue.cc:260] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:37.108268 16331 ts_tablet_manager.cc:1428] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:37:37.108651 16314 heartbeater.cc:503] Master 127.30.146.190:33317 was elected leader, sending a full tablet report...
I20260808 10:37:37.108639 16171 heartbeater.cc:503] Master 127.30.146.190:33317 was elected leader, sending a full tablet report...
I20260808 10:37:37.108788 16333 ts_tablet_manager.cc:1428] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:37:37.109659 16020 heartbeater.cc:503] Master 127.30.146.190:33317 was elected leader, sending a full tablet report...
I20260808 10:37:37.110203 16332 ts_tablet_manager.cc:1428] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
W20260808 10:37:37.273341 16315 tablet.cc:2369] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:37:37.294953 16172 tablet.cc:2369] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:37:37.307317 16023 tablet.cc:2369] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:37:37.370098 16337 raft_consensus.cc:482] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:37:37.370335 16337 raft_consensus.cc:504] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:37.371634 16337 leader_election.cc:288] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111), 9310e650d22e4d3889b6186f9f3a2247 (127.30.146.129:46423)
I20260808 10:37:37.376096 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.371997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:59089 (local address 127.30.146.131:39111)
0808 10:37:37.372152 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:37.372156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:37.372411 (+ 255us) server_negotiation.cc:407] Connection header received
0808 10:37:37.372777 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:37.372780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:37.372821 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:37.372882 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:37.373204 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.373930 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.375046 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.375477 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.375612 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.375666 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:37.375872 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 10:37:37.375945 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:37:37.376108 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.372006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:34911 (local address 127.30.146.129:46423)
0808 10:37:37.372247 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:37.372251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:37.372382 (+ 131us) server_negotiation.cc:407] Connection header received
0808 10:37:37.372542 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:37.372545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:37.372588 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:37.372657 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:37.372995 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.373851 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.375041 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.375488 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.375618 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.375680 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:37.375906 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 10:37:37.375981 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260808 10:37:37.376127 16341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.371879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:46423 (local address 127.30.146.130:34911)
0808 10:37:37.372285 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:37.372299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:37.372437 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:37.372684 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:37.372686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:37.372702 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:37.372896 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.372906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.373985 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.373989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.374909 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.374919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.375619 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.375622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.375780 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.375799 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:37:37.375887 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:37:37.375966 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":61,"spinlock_wait_cycles":22656,"thread_start_us":80,"threads_started":1}
I20260808 10:37:37.376143 16340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.371879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:39111 (local address 127.30.146.130:59089)
0808 10:37:37.372269 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:37.372286 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:37.372437 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:37.372923 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:37.372927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:37.372948 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:37.373076 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.373081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.374034 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.374038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.374914 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:37.374925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.375612 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.375615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:37.375722 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.375764 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 10:37:37.375854 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 10:37:37.375934 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":23,"spinlock_wait_cycles":12416,"thread_start_us":81,"threads_started":1}
I20260808 10:37:37.376760 15972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6729c3d1f46044e59a6227c447ce6778" candidate_uuid: "e02f043225a1467c98355320ed4ce356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9310e650d22e4d3889b6186f9f3a2247" is_pre_election: true
I20260808 10:37:37.377099 15972 raft_consensus.cc:2454] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e02f043225a1467c98355320ed4ce356 in term 0.
I20260808 10:37:37.377230 16263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6729c3d1f46044e59a6227c447ce6778" candidate_uuid: "e02f043225a1467c98355320ed4ce356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "662378f8eb92444f9ed9d9a6b87638b4" is_pre_election: true
I20260808 10:37:37.377573 16263 raft_consensus.cc:2454] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e02f043225a1467c98355320ed4ce356 in term 0.
I20260808 10:37:37.377736 16047 leader_election.cc:302] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9310e650d22e4d3889b6186f9f3a2247, e02f043225a1467c98355320ed4ce356; no voters:
I20260808 10:37:37.378041 16337 raft_consensus.cc:2781] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:37:37.378131 16337 raft_consensus.cc:482] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:37:37.378162 16337 raft_consensus.cc:3037] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:37.379309 16337 raft_consensus.cc:504] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:37.379738 16337 leader_election.cc:288] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [CANDIDATE]: Term 1 election: Requested vote from peers 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111), 9310e650d22e4d3889b6186f9f3a2247 (127.30.146.129:46423)
I20260808 10:37:37.380131 16263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6729c3d1f46044e59a6227c447ce6778" candidate_uuid: "e02f043225a1467c98355320ed4ce356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "662378f8eb92444f9ed9d9a6b87638b4"
I20260808 10:37:37.380244 15972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6729c3d1f46044e59a6227c447ce6778" candidate_uuid: "e02f043225a1467c98355320ed4ce356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9310e650d22e4d3889b6186f9f3a2247"
I20260808 10:37:37.380297 16263 raft_consensus.cc:3037] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:37.380398 15972 raft_consensus.cc:3037] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:37.381992 15972 raft_consensus.cc:2454] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e02f043225a1467c98355320ed4ce356 in term 1.
I20260808 10:37:37.381992 16263 raft_consensus.cc:2454] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e02f043225a1467c98355320ed4ce356 in term 1.
I20260808 10:37:37.382352 16047 leader_election.cc:302] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9310e650d22e4d3889b6186f9f3a2247, e02f043225a1467c98355320ed4ce356; no voters:
I20260808 10:37:37.382592 16337 raft_consensus.cc:2781] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:37:37.382854 16337 raft_consensus.cc:686] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 1 LEADER]: Becoming Leader. State: Replica: e02f043225a1467c98355320ed4ce356, State: Running, Role: LEADER
I20260808 10:37:37.383204 16337 consensus_queue.cc:237] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:37.386701 15817 catalog_manager.cc:5495] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 reported cstate change: term changed from 0 to 1, leader changed from <none> to e02f043225a1467c98355320ed4ce356 (127.30.146.130). New cstate: current_term: 1 leader_uuid: "e02f043225a1467c98355320ed4ce356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } health_report { overall_health: HEALTHY } } }
I20260808 10:37:37.448750 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.444830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59484 (local address 127.30.146.130:38867)
0808 10:37:37.444994 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:37.444998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:37.445041 (+ 43us) server_negotiation.cc:407] Connection header received
0808 10:37:37.445119 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:37.445122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:37.445202 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:37.445294 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:37:37.445594 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.446104 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.447120 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:37.447354 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:37.447449 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:37.447817 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:37:37.448239 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:37:37.448333 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:37.448453 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 10:37:37.448508 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:37:37.455195 16263 raft_consensus.cc:1261] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4 [term 1 FOLLOWER]: Refusing update from remote peer e02f043225a1467c98355320ed4ce356: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:37.455197 15972 raft_consensus.cc:1261] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Refusing update from remote peer e02f043225a1467c98355320ed4ce356: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:37.455976 16337 consensus_queue.cc:1035] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [LEADER]: Connected to new peer: Peer: permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:37.456249 16344 consensus_queue.cc:1035] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:37.465962 16349 mvcc.cc:205] Tried to move back new op lower bound from 7316215633728720896 to 7316215633444524032. Current Snapshot: MvccSnapshot[applied={T|T < 7316215633728720896}]
I20260808 10:37:37.474529 15817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:37:37.474767 15817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:37:37.477576 15817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Sending DeleteTablet for 3 replicas of tablet 6729c3d1f46044e59a6227c447ce6778
I20260808 10:37:37.478540 16242 tablet_service.cc:1514] Processing DeleteTablet for tablet 6729c3d1f46044e59a6227c447ce6778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:37 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:46002
I20260808 10:37:37.478600 16099 tablet_service.cc:1514] Processing DeleteTablet for tablet 6729c3d1f46044e59a6227c447ce6778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:37 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:59470
I20260808 10:37:37.478777 15951 tablet_service.cc:1514] Processing DeleteTablet for tablet 6729c3d1f46044e59a6227c447ce6778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:37 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:44498
I20260808 10:37:37.479238 16358 tablet_replica.cc:330] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: stopping tablet replica
I20260808 10:37:37.479518 16360 tablet_replica.cc:330] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: stopping tablet replica
I20260808 10:37:37.479521 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 15886
I20260808 10:37:37.479908 16360 raft_consensus.cc:2229] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:37.482358 16359 tablet_replica.cc:330] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: stopping tablet replica
I20260808 10:37:37.482669 16359 raft_consensus.cc:2229] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:37:37.483079 16359 raft_consensus.cc:2258] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:37.483605 16358 raft_consensus.cc:2229] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:37.483978 16358 raft_consensus.cc:2258] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:37.485018 16359 ts_tablet_manager.cc:1905] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:37.485639 16358 ts_tablet_manager.cc:1905] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:37.488372 16359 ts_tablet_manager.cc:1918] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:37.488579 16359 log.cc:1196] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/6729c3d1f46044e59a6227c447ce6778
I20260808 10:37:37.488637 16358 ts_tablet_manager.cc:1918] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:37.488744 16358 log.cc:1196] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/6729c3d1f46044e59a6227c447ce6778
I20260808 10:37:37.489188 16358 ts_tablet_manager.cc:1939] T 6729c3d1f46044e59a6227c447ce6778 P 662378f8eb92444f9ed9d9a6b87638b4: Deleting consensus metadata
I20260808 10:37:37.489184 16359 ts_tablet_manager.cc:1939] T 6729c3d1f46044e59a6227c447ce6778 P e02f043225a1467c98355320ed4ce356: Deleting consensus metadata
I20260808 10:37:37.490556 15802 catalog_manager.cc:4841] TS e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867): tablet 6729c3d1f46044e59a6227c447ce6778 (table test-table [id=58c6e4aff8474d09acfd995045607fd9]) successfully deleted
I20260808 10:37:37.490984 15804 catalog_manager.cc:4841] TS 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111): tablet 6729c3d1f46044e59a6227c447ce6778 (table test-table [id=58c6e4aff8474d09acfd995045607fd9]) successfully deleted
W20260808 10:37:37.491613 15801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:46423 (error 108)
I20260808 10:37:37.491884 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 16029
I20260808 10:37:37.494894 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:37.494514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:46423 (local address 127.0.0.1:44502)
0808 10:37:37.494746 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:37.494809 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:46423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260808 10:37:37.495077 15801 catalog_manager.cc:4584] TS 9310e650d22e4d3889b6186f9f3a2247 (127.30.146.129:46423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6729c3d1f46044e59a6227c447ce6778: Network error: Client connection negotiation failed: client connection to 127.30.146.129:46423: connect: Connection refused (error 111)
I20260808 10:37:37.501329 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 16176
I20260808 10:37:37.509829 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 15783
I20260808 10:37:57.519371 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33317
--webserver_interface=127.30.146.190
--webserver_port=42431
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:33317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:37:57.635998 16362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:57.636387 16362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:57.636433 16362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:57.640452 16362 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:37:57.640512 16362 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:37:57.640528 16362 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:37:57.640543 16362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:37:57.640561 16362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:57.640580 16362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:57.640595 16362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:37:57.640610 16362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:37:57.645943 16362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:33317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=42431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:57.647054 16362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:57.648033 16362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:57.662207 16368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:57.662223 16369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:57.662184 16371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:57.662595 16362 server_base.cc:1034] running on GCE node
I20260808 10:37:57.663582 16362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:57.664721 16362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:57.665912 16362 hybrid_clock.cc:648] HybridClock initialized: now 1786185477665889 us; error 37 us; skew 500 ppm
Aug 08 10:37:57 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185477, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:57.669101 16362 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:37:57.670157 16362 webserver.cc:460] Webserver started at http://127.30.146.190:42431/ using document root <none> and password file <none>
I20260808 10:37:57.670749 16362 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:57.670828 16362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:57.675287 16362 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:37:57.677749 16379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:57.679177 16362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:57.679296 16362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "b6b79a18f3894a5486b416a4d8ea5641"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "f9a38043a24c097ad6bcbf4d1e800db3"
server_key_iv: "f8bca7e6c6c667eb849b3c4f9deaede9"
server_key_version: "encryptionkey@0"
I20260808 10:37:57.679944 16362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:57.694687 16362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:57.698158 16362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:57.698356 16362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:57.709189 16362 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:33317
I20260808 10:37:57.709522 16435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:33317 every 8 connection(s)
I20260808 10:37:57.710634 16362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:37:57.715349 16436 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:37:57.717185 16436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Bootstrap starting.
I20260808 10:37:57.717806 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 16362
I20260808 10:37:57.718320 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:46423
--local_ip_for_outbound_sockets=127.30.146.129
--tserver_master_addrs=127.30.146.190:33317
--webserver_port=46075
--webserver_interface=127.30.146.129
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:37:57.736249 16436 log.cc:824] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:57.745249 16436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:37:57.745873 16436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Bootstrap complete.
I20260808 10:37:57.750772 16436 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:37:57.751251 16436 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b79a18f3894a5486b416a4d8ea5641, State: Initialized, Role: FOLLOWER
I20260808 10:37:57.751817 16436 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:37:57.751996 16436 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:37:57.752087 16436 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:37:57.752202 16436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:37:57.754544 16436 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:37:57.754895 16436 leader_election.cc:302] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6b79a18f3894a5486b416a4d8ea5641; no voters:
I20260808 10:37:57.755249 16436 leader_election.cc:288] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:37:57.755391 16440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:37:57.755640 16440 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 2 LEADER]: Becoming Leader. State: Replica: b6b79a18f3894a5486b416a4d8ea5641, State: Running, Role: LEADER
I20260808 10:37:57.756002 16440 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:37:57.756601 16436 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:37:57.757155 16440 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6b79a18f3894a5486b416a4d8ea5641. Latest consensus state: current_term: 2 leader_uuid: "b6b79a18f3894a5486b416a4d8ea5641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } } }
I20260808 10:37:57.757361 16440 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:57.757606 16441 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b6b79a18f3894a5486b416a4d8ea5641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } } }
I20260808 10:37:57.757732 16441 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: This master's current role is: LEADER
I20260808 10:37:57.757948 16448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:37:57.762846 16448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:37:57.763613 16448 catalog_manager.cc:1259] Loaded cluster ID: f2be7bf1d1f64167bf0235ffb3f2f36f
I20260808 10:37:57.763703 16448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:37:57.765734 16448 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:37:57.766350 16448 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Loaded TSK: 0
I20260808 10:37:57.781805 16448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Generated new TSK 1
I20260808 10:37:57.782099 16448 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:37:57.858316 16438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:57.858668 16438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:57.858754 16438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:57.863039 16438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:57.863135 16438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:57.863240 16438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:37:57.868865 16438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:46423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=46075
--tserver_master_addrs=127.30.146.190:33317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:57.869988 16438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:57.871070 16438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:57.882438 16468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:57.882431 16465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:57.882431 16466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:57.883174 16438 server_base.cc:1034] running on GCE node
I20260808 10:37:57.883616 16438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:57.884446 16438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:57.885644 16438 hybrid_clock.cc:648] HybridClock initialized: now 1786185477885610 us; error 42 us; skew 500 ppm
Aug 08 10:37:57 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185477, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:57.888537 16438 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:37:57.889588 16438 webserver.cc:460] Webserver started at http://127.30.146.129:46075/ using document root <none> and password file <none>
I20260808 10:37:57.890165 16438 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:57.890244 16438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:57.894315 16438 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:37:57.896780 16475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:57.898168 16438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:37:57.898293 16438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "9310e650d22e4d3889b6186f9f3a2247"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "d972e00e2f48a0172d6fa810d64f651c"
server_key_iv: "f7b57aa37a074af82046d4b6fecca154"
server_key_version: "encryptionkey@0"
I20260808 10:37:57.898861 16438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:57.911137 16438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:57.914726 16438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:57.914935 16438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:57.915619 16438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:57.916872 16482 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:37:57.921190 16438 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:37:57.921252 16438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 10:37:57.921324 16438 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:37:57.923077 16438 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:37:57.923135 16438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 10:37:57.923401 16482 tablet_bootstrap.cc:492] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Bootstrap starting.
I20260808 10:37:57.934850 16438 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:46423
I20260808 10:37:57.934870 16597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:46423 every 8 connection(s)
I20260808 10:37:57.936156 16438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:37:57.936619 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 16438
I20260808 10:37:57.937192 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:38867
--local_ip_for_outbound_sockets=127.30.146.130
--tserver_master_addrs=127.30.146.190:33317
--webserver_port=45757
--webserver_interface=127.30.146.130
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:37:57.946563 16482 log.cc:824] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:37:57 dist-test-slave-9gcw krb5kdc[15769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185477, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:57.955425 16482 tablet_bootstrap.cc:492] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:37:57.955914 16482 tablet_bootstrap.cc:492] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Bootstrap complete.
I20260808 10:37:57.957536 16482 ts_tablet_manager.cc:1397] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Time spent bootstrapping tablet: real 0.034s user 0.022s sys 0.012s
I20260808 10:37:57.958370 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:57.938975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:54221 (local address 127.30.146.190:33317)
0808 10:37:57.939565 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:57.939588 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:57.940166 (+ 578us) server_negotiation.cc:407] Connection header received
0808 10:37:57.941780 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:57.941796 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:57.942206 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:57.942629 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:57.944411 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:57.945930 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:57.947916 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:57.948632 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:57.948810 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:57.952153 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:57.952160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:57.952179 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:57.952216 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:57.954425 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:57.955052 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:57.955058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:57.955063 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:57.955220 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:57.955649 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:57.955751 (+ 102us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:57.955754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:57.956248 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:57.956386 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:57.957064 (+ 678us) server_negotiation.cc:299] Negotiation successful
0808 10:37:57.957420 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":136,"threads_started":1}
I20260808 10:37:57.958398 16602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:57.938929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:54221)
0808 10:37:57.939935 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:57.940000 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:57.941291 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:57.942862 (+ 1571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:57.942873 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:57.943437 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:57.944198 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:57.944224 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:57.946114 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:57.946118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:57.947765 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:57.947782 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:57.948819 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:57.948826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:57.949066 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:57.949880 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:57.949908 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:57.951938 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:57.954539 (+ 2601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:57.954546 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:57.954559 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:57.954915 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:57.955336 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:57.955339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:57.955341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:57.955508 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:57.956383 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:57.956390 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:57.956681 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 10:37:57.956930 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 10:37:57.957372 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":155,"threads_started":1}
I20260808 10:37:57.960347 16482 raft_consensus.cc:348] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:57.960938 16482 raft_consensus.cc:729] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9310e650d22e4d3889b6186f9f3a2247, State: Initialized, Role: FOLLOWER
I20260808 10:37:57.961328 16598 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33317
I20260808 10:37:57.961582 16598 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:57.961513 16482 consensus_queue.cc:260] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } }
I20260808 10:37:57.962184 16598 heartbeater.cc:511] Master 127.30.146.190:33317 requested a full tablet report, sending...
I20260808 10:37:57.962579 16482 ts_tablet_manager.cc:1428] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:37:57.964955 16397 ts_manager.cc:194] Registered new tserver with Master: 9310e650d22e4d3889b6186f9f3a2247 (127.30.146.129:46423)
I20260808 10:37:57.968365 16397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:54221
I20260808 10:37:57.971253 16598 heartbeater.cc:503] Master 127.30.146.190:33317 was elected leader, sending a full tablet report...
Aug 08 10:37:57 dist-test-slave-9gcw krb5kdc[15769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185477, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.129@KRBTEST.COM
I20260808 10:37:57.979193 16607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:57.967894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54290 (local address 127.30.146.129:46423)
0808 10:37:57.968287 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:57.968300 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:57.968697 (+ 397us) server_negotiation.cc:407] Connection header received
0808 10:37:57.968967 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:57.968974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:57.969223 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:57.969334 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:57.970121 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:57.970871 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:57.971904 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:57.972269 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:57.972411 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:57.974935 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:57.974944 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:57.974955 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:57.974977 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:57.976972 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:57.977606 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:57.977611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:57.977612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:57.977702 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:57.978025 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:57.978030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:57.978032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:57.978298 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:57.978454 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:57.978827 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 10:37:57.979002 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":302,"thread_start_us":112,"threads_started":1}
I20260808 10:37:57.979193 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:57.967820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:46423 (local address 127.0.0.1:54290)
0808 10:37:57.968561 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:57.968604 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:57.968798 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:57.969552 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:57.969556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:57.969892 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:57.969994 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:57.970000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:57.970998 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:57.971002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:57.971780 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:57.971791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:57.972414 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:57.972418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:57.972534 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:57.973138 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:57.973175 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:57.974737 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:57.977114 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:57.977125 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:57.977131 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:57.977446 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:57.977820 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:57.977824 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:57.977826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:57.977922 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:57.978442 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:57.978448 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:57.978587 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 10:37:57.978740 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 10:37:57.978981 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":133,"threads_started":1}
I20260808 10:37:57.980353 16528 tablet_service.cc:1514] Processing DeleteTablet for tablet 6729c3d1f46044e59a6227c447ce6778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:37 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:54290
I20260808 10:37:57.981424 16608 tablet_replica.cc:330] stopping tablet replica
I20260808 10:37:57.981772 16608 raft_consensus.cc:2229] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:57.982038 16608 raft_consensus.cc:2258] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:57.983023 16608 ts_tablet_manager.cc:1905] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:57.987375 16608 ts_tablet_manager.cc:1918] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:57.987509 16608 log.cc:1196] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal/wals/6729c3d1f46044e59a6227c447ce6778
I20260808 10:37:57.987828 16608 ts_tablet_manager.cc:1939] T 6729c3d1f46044e59a6227c447ce6778 P 9310e650d22e4d3889b6186f9f3a2247: Deleting consensus metadata
I20260808 10:37:57.990036 16380 catalog_manager.cc:4841] TS 9310e650d22e4d3889b6186f9f3a2247 (127.30.146.129:46423): tablet 6729c3d1f46044e59a6227c447ce6778 (table test-table [id=58c6e4aff8474d09acfd995045607fd9]) successfully deleted
W20260808 10:37:58.065630 16600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:58.065927 16600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:58.065973 16600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:58.069977 16600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:58.070039 16600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:58.070103 16600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:37:58.075713 16600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:38867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=45757
--tserver_master_addrs=127.30.146.190:33317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:58.076911 16600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:58.078037 16600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:58.088227 16613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:58.088210 16616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:58.088305 16614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:58.088554 16600 server_base.cc:1034] running on GCE node
I20260808 10:37:58.089015 16600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:58.089771 16600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:58.090986 16600 hybrid_clock.cc:648] HybridClock initialized: now 1786185478090973 us; error 27 us; skew 500 ppm
Aug 08 10:37:58 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185478, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:58.094197 16600 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:37:58.095299 16600 webserver.cc:460] Webserver started at http://127.30.146.130:45757/ using document root <none> and password file <none>
I20260808 10:37:58.096022 16600 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:58.096145 16600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:58.100451 16600 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:37:58.102918 16623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:58.104290 16600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:37:58.104436 16600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "e02f043225a1467c98355320ed4ce356"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "f5673d70d7ffab605ec5b1ad92480006"
server_key_iv: "2500cc6151dee5de61e24cf751fcb4bd"
server_key_version: "encryptionkey@0"
I20260808 10:37:58.105047 16600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:58.128193 16600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:58.131665 16600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:58.131891 16600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:58.132558 16600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:58.133472 16600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:58.133522 16600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:58.133592 16600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:58.133633 16600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:58.144100 16600 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:38867
I20260808 10:37:58.144126 16744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:38867 every 8 connection(s)
I20260808 10:37:58.145179 16600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:37:58.146175 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 16600
I20260808 10:37:58.146746 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:39111
--local_ip_for_outbound_sockets=127.30.146.131
--tserver_master_addrs=127.30.146.190:33317
--webserver_port=42747
--webserver_interface=127.30.146.131
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:37:58 dist-test-slave-9gcw krb5kdc[15769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185478, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:58.163312 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.147479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:49227 (local address 127.30.146.190:33317)
0808 10:37:58.147653 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:58.147659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:58.148977 (+ 1318us) server_negotiation.cc:407] Connection header received
0808 10:37:58.150032 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:58.150035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:58.150106 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:58.150215 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:58.152163 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.152805 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.154921 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.155192 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.155289 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.159055 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:58.159059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:58.159061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:58.159089 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:58.160857 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:58.161548 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:58.161551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:58.161552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:58.161646 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:58.162032 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:58.162035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:58.162037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:58.162244 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:58.162366 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:58.162962 (+ 596us) server_negotiation.cc:299] Negotiation successful
0808 10:37:58.163112 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:37:58.164166 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.147903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:49227)
0808 10:37:58.148767 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:58.148835 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:58.149795 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:58.150446 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:58.150459 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:58.151139 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:58.151914 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.151974 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.153031 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.153037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.154706 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.154721 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.155405 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.155412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.155681 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.156461 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:58.156490 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:58.158851 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:58.161000 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:58.161007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:58.161024 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:58.161415 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:58.161751 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:58.161755 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:58.161757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:58.161915 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:58.162364 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:58.162372 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:58.162695 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 10:37:58.162897 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 10:37:58.163273 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":125,"threads_started":1}
I20260808 10:37:58.165521 16745 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33317
I20260808 10:37:58.165810 16745 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:58.166402 16745 heartbeater.cc:511] Master 127.30.146.190:33317 requested a full tablet report, sending...
I20260808 10:37:58.167717 16397 ts_manager.cc:194] Registered new tserver with Master: e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867)
I20260808 10:37:58.168375 16397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:49227
W20260808 10:37:58.272910 16748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:37:58.273242 16748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:37:58.273348 16748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:37:58.277549 16748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:37:58.277608 16748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:37:58.277668 16748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:37:58.283347 16748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:39111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=42747
--tserver_master_addrs=127.30.146.190:33317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:37:58.284693 16748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:37:58.285812 16748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:37:58.296468 16754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:58.296430 16758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:37:58.296432 16755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:37:58.296509 16748 server_base.cc:1034] running on GCE node
I20260808 10:37:58.297398 16748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:37:58.298173 16748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:37:58.299393 16748 hybrid_clock.cc:648] HybridClock initialized: now 1786185478299361 us; error 55 us; skew 500 ppm
Aug 08 10:37:58 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185478, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:37:58.302893 16748 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:37:58.303953 16748 webserver.cc:460] Webserver started at http://127.30.146.131:42747/ using document root <none> and password file <none>
I20260808 10:37:58.304608 16748 fs_manager.cc:359] Metadata directory not provided
I20260808 10:37:58.304751 16748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:37:58.309468 16748 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:58.311775 16765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:58.313290 16748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:37:58.313426 16748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "662378f8eb92444f9ed9d9a6b87638b4"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "4bc2bfb82cdfa88a194f284d6dca2dd1"
server_key_iv: "2e530e075049d5441e5993d282477e9e"
server_key_version: "encryptionkey@0"
I20260808 10:37:58.313997 16748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:37:58.330518 16748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:37:58.334491 16748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:37:58.334748 16748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:37:58.335461 16748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:37:58.336429 16748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:37:58.336508 16748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:58.336594 16748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:37:58.336644 16748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:37:58.347419 16748 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:39111
I20260808 10:37:58.347453 16887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:39111 every 8 connection(s)
I20260808 10:37:58.348541 16748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:37:58.356001 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 16748
Aug 08 10:37:58 dist-test-slave-9gcw krb5kdc[15769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185478, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:37:58.366331 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.350632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:48151 (local address 127.30.146.190:33317)
0808 10:37:58.350853 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:58.350857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:58.351912 (+ 1055us) server_negotiation.cc:407] Connection header received
0808 10:37:58.353048 (+ 1136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:58.353052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:58.353105 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:58.353311 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:58.354734 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.355404 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.357051 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.357542 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.357653 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.361427 (+ 3774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:58.361431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:58.361434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:58.361462 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:58.363305 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:58.364209 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:58.364212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:58.364213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:58.364303 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:58.364780 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:58.364787 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:58.364793 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:58.365009 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:58.365102 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:58.365951 (+ 849us) server_negotiation.cc:299] Negotiation successful
0808 10:37:58.366119 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260808 10:37:58.367436 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.350984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:48151)
0808 10:37:58.351741 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:58.351788 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:58.352800 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:58.353526 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:58.353534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:58.354006 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:37:58.354565 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.354580 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.355573 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.355577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.356860 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.356876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.357917 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.357924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.358201 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.359074 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:37:58.359096 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:37:58.361236 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:37:58.363559 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:58.363565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:58.363583 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:58.364078 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:58.364420 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:37:58.364427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:37:58.364432 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:37:58.364656 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:37:58.365167 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:37:58.365175 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:37:58.365539 (+ 364us) client_negotiation.cc:769] Sending connection context
0808 10:37:58.365865 (+ 326us) client_negotiation.cc:240] Negotiation successful
0808 10:37:58.366165 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":157,"threads_started":1}
I20260808 10:37:58.367584 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.356801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.30.146.190:33317)
0808 10:37:58.357111 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:58.357116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:58.357299 (+ 183us) server_negotiation.cc:407] Connection header received
0808 10:37:58.357449 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:58.357453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:58.357510 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:58.357631 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:58.358745 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.359636 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.361056 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.361481 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.361567 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.363574 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:58.363578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:58.363581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:58.363608 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:58.365660 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:58.366249 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:58.366252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:58.366254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:58.366347 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:58.366691 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:58.366697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:58.366699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:58.366886 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:58.367057 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:58.367310 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 10:37:58.367424 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":110,"threads_started":1}
I20260808 10:37:58.368927 16888 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33317
I20260808 10:37:58.369282 16888 heartbeater.cc:464] Registering TS with master...
I20260808 10:37:58.369827 16888 heartbeater.cc:511] Master 127.30.146.190:33317 requested a full tablet report, sending...
I20260808 10:37:58.370878 16396 ts_manager.cc:194] Registered new tserver with Master: 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111)
I20260808 10:37:58.371520 16396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:48151
I20260808 10:37:58.382133 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:37:58.386569 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.382797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47642 (local address 127.30.146.190:33317)
0808 10:37:58.382936 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:58.382939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:58.383104 (+ 165us) server_negotiation.cc:407] Connection header received
0808 10:37:58.383252 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:58.383254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:58.383354 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:58.383441 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:37:58.383789 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.384376 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.385463 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.385740 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.385825 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.386116 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:37:58.386255 (+ 139us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 10:37:58.386464 (+ 209us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47642: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 10:37:58.386657 16893 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47642: token expired
W20260808 10:37:58.387048 16318 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.146.190:33317)
I20260808 10:37:58.394656 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.387449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47648 (local address 127.30.146.190:33317)
0808 10:37:58.387580 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:58.387586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:58.387603 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:37:58.387805 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:58.387808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:58.387853 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:58.387940 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:37:58.388569 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.389046 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.389882 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.390101 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.390223 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.391569 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:37:58.391577 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:37:58.391579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:37:58.391601 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:37:58.392926 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:58.393429 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:58.393432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:58.393434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:58.393516 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:37:58.393812 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:37:58.393815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:37:58.393817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:37:58.393991 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:37:58.394127 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:58.394367 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 10:37:58.394490 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:37:58.397406 16396 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:37:58.400694 16396 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:37:58.408838 16528 tablet_service.cc:1467] Processing CreateTablet for tablet db92504b3bfe4b37a2c20a892a289df7 (DEFAULT_TABLE table=test-table [id=226f315ca4f849c3a6cd4ad0a51c915c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:58.409607 16528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db92504b3bfe4b37a2c20a892a289df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:58.412905 16482 tablet_bootstrap.cc:492] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Bootstrap starting.
I20260808 10:37:58.413053 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.407742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:39111 (local address 127.0.0.1:47344)
0808 10:37:58.407906 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:58.407921 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:58.408002 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:58.408902 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:58.408905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:58.408923 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:58.409041 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.409049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.410541 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.410544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.411757 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.411766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.412591 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.412595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.412710 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.412751 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 10:37:58.412828 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:37:58.412910 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":1}
I20260808 10:37:58.413622 16899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.407918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.30.146.131:39111)
0808 10:37:58.408346 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:58.408372 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:58.408397 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:37:58.408482 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:58.408490 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:58.408747 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:58.408924 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:58.409213 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.410407 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.411874 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.412466 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.412607 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.412749 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:58.413353 (+ 604us) server_negotiation.cc:299] Negotiation successful
0808 10:37:58.413465 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":145,"threads_started":1}
I20260808 10:37:58.413934 16482 tablet_bootstrap.cc:654] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:58.414656 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.408322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.30.146.130:38867)
0808 10:37:58.409298 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:58.409311 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:58.409333 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:37:58.409409 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:58.409416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:58.409601 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:58.409747 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:58.410066 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.411452 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.412878 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.414093 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.414237 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.414366 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:58.414418 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 10:37:58.414500 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":157,"threads_started":1}
I20260808 10:37:58.415510 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.407917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:38867 (local address 127.0.0.1:47804)
0808 10:37:58.408754 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:58.408769 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:58.408868 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:58.409823 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:58.409827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:58.409846 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:58.409945 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.409951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.411615 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.411621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.412736 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.412749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.414223 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.414227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.414308 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.414329 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:37:58.414395 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:37:58.414463 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":99,"threads_started":1}
I20260808 10:37:58.415772 16482 tablet_bootstrap.cc:492] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: No bootstrap required, opened a new log
I20260808 10:37:58.415866 16482 ts_tablet_manager.cc:1397] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:37:58.416337 16482 raft_consensus.cc:348] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:37:58.416477 16482 raft_consensus.cc:374] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:58.416510 16482 raft_consensus.cc:729] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9310e650d22e4d3889b6186f9f3a2247, State: Initialized, Role: FOLLOWER
I20260808 10:37:58.416646 16482 consensus_queue.cc:260] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:37:58.416930 16482 ts_tablet_manager.cc:1428] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:37:58.416920 16816 tablet_service.cc:1467] Processing CreateTablet for tablet db92504b3bfe4b37a2c20a892a289df7 (DEFAULT_TABLE table=test-table [id=226f315ca4f849c3a6cd4ad0a51c915c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:58.418134 16816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db92504b3bfe4b37a2c20a892a289df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:58.418393 16675 tablet_service.cc:1467] Processing CreateTablet for tablet db92504b3bfe4b37a2c20a892a289df7 (DEFAULT_TABLE table=test-table [id=226f315ca4f849c3a6cd4ad0a51c915c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:37:58.419381 16675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db92504b3bfe4b37a2c20a892a289df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:37:58.426748 16903 tablet_bootstrap.cc:492] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: Bootstrap starting.
I20260808 10:37:58.427749 16904 tablet_bootstrap.cc:492] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: Bootstrap starting.
I20260808 10:37:58.430178 16903 tablet_bootstrap.cc:654] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:58.430881 16904 tablet_bootstrap.cc:654] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: Neither blocks nor log segments found. Creating new log.
I20260808 10:37:58.431329 16903 log.cc:824] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:58.431828 16904 log.cc:824] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: Log is configured to *not* fsync() on all Append() calls
I20260808 10:37:58.434002 16903 tablet_bootstrap.cc:492] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: No bootstrap required, opened a new log
I20260808 10:37:58.433995 16904 tablet_bootstrap.cc:492] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: No bootstrap required, opened a new log
I20260808 10:37:58.434219 16903 ts_tablet_manager.cc:1397] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 10:37:58.434219 16904 ts_tablet_manager.cc:1397] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
W20260808 10:37:58.437837 16599 tablet.cc:2369] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:37:58.438411 16904 raft_consensus.cc:348] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:37:58.438411 16903 raft_consensus.cc:348] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:37:58.438730 16903 raft_consensus.cc:374] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:58.438730 16904 raft_consensus.cc:374] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:37:58.438786 16903 raft_consensus.cc:729] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 662378f8eb92444f9ed9d9a6b87638b4, State: Initialized, Role: FOLLOWER
I20260808 10:37:58.438794 16904 raft_consensus.cc:729] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02f043225a1467c98355320ed4ce356, State: Initialized, Role: FOLLOWER
I20260808 10:37:58.439285 16903 consensus_queue.cc:260] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:37:58.439285 16904 consensus_queue.cc:260] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:37:58.440202 16745 heartbeater.cc:503] Master 127.30.146.190:33317 was elected leader, sending a full tablet report...
I20260808 10:37:58.440202 16888 heartbeater.cc:503] Master 127.30.146.190:33317 was elected leader, sending a full tablet report...
I20260808 10:37:58.440369 16904 ts_tablet_manager.cc:1428] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:37:58.440369 16903 ts_tablet_manager.cc:1428] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 10:37:58.599831 16889 tablet.cc:2369] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:37:58.646622 16746 tablet.cc:2369] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:37:58.652612 16605 raft_consensus.cc:482] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:37:58.652949 16605 raft_consensus.cc:504] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:37:58.654342 16605 leader_election.cc:288] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111), e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867)
I20260808 10:37:58.660070 16899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.654736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:53947 (local address 127.30.146.131:39111)
0808 10:37:58.654895 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:58.654902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:58.655009 (+ 107us) server_negotiation.cc:407] Connection header received
0808 10:37:58.655233 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:58.655237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:58.655293 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:58.655401 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:58.655677 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.656594 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.658499 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.659220 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.659365 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.659425 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:58.659826 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 10:37:58.659927 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:37:58.660146 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.654546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:39111 (local address 127.30.146.129:53947)
0808 10:37:58.654893 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:58.654911 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:58.655084 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:58.655413 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:58.655416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:58.655434 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:58.655551 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.655557 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.656743 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.656746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.658368 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.658373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.659398 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.659403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.659610 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.659677 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 10:37:58.659787 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:37:58.659923 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":229,"thread_start_us":82,"threads_started":1}
I20260808 10:37:58.660347 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.654777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:56955 (local address 127.30.146.130:38867)
0808 10:37:58.654916 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:58.654921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:58.655150 (+ 229us) server_negotiation.cc:407] Connection header received
0808 10:37:58.655333 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:58.655341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:58.655388 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:58.655470 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:37:58.655832 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.656659 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.658499 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.659220 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.659365 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.659425 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:58.660074 (+ 649us) server_negotiation.cc:299] Negotiation successful
0808 10:37:58.660208 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:37:58.660545 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.654664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:38867 (local address 127.30.146.129:56955)
0808 10:37:58.655043 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:58.655055 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:37:58.655179 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:37:58.655535 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:37:58.655538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:37:58.655561 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:37:58.655700 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.655707 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.656804 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.656807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.658359 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:37:58.658371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.659398 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.659403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:37:58.659694 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.659724 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 10:37:58.659804 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:37:58.659923 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":200,"thread_start_us":67,"threads_started":1}
I20260808 10:37:58.661010 16837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db92504b3bfe4b37a2c20a892a289df7" candidate_uuid: "9310e650d22e4d3889b6186f9f3a2247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "662378f8eb92444f9ed9d9a6b87638b4" is_pre_election: true
I20260808 10:37:58.661299 16697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db92504b3bfe4b37a2c20a892a289df7" candidate_uuid: "9310e650d22e4d3889b6186f9f3a2247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02f043225a1467c98355320ed4ce356" is_pre_election: true
I20260808 10:37:58.661389 16837 raft_consensus.cc:2454] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9310e650d22e4d3889b6186f9f3a2247 in term 0.
I20260808 10:37:58.661593 16697 raft_consensus.cc:2454] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9310e650d22e4d3889b6186f9f3a2247 in term 0.
I20260808 10:37:58.661943 16479 leader_election.cc:302] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 662378f8eb92444f9ed9d9a6b87638b4, 9310e650d22e4d3889b6186f9f3a2247; no voters:
I20260808 10:37:58.662325 16605 raft_consensus.cc:2781] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:37:58.662424 16605 raft_consensus.cc:482] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:37:58.662473 16605 raft_consensus.cc:3037] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:58.663594 16605 raft_consensus.cc:504] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:37:58.664029 16605 leader_election.cc:288] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 1 election: Requested vote from peers 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111), e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867)
I20260808 10:37:58.664448 16837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db92504b3bfe4b37a2c20a892a289df7" candidate_uuid: "9310e650d22e4d3889b6186f9f3a2247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "662378f8eb92444f9ed9d9a6b87638b4"
I20260808 10:37:58.664496 16697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db92504b3bfe4b37a2c20a892a289df7" candidate_uuid: "9310e650d22e4d3889b6186f9f3a2247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02f043225a1467c98355320ed4ce356"
I20260808 10:37:58.664623 16697 raft_consensus.cc:3037] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:58.664647 16837 raft_consensus.cc:3037] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:37:58.666174 16697 raft_consensus.cc:2454] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9310e650d22e4d3889b6186f9f3a2247 in term 1.
I20260808 10:37:58.666185 16837 raft_consensus.cc:2454] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9310e650d22e4d3889b6186f9f3a2247 in term 1.
I20260808 10:37:58.666548 16477 leader_election.cc:302] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9310e650d22e4d3889b6186f9f3a2247, e02f043225a1467c98355320ed4ce356; no voters:
I20260808 10:37:58.666836 16605 raft_consensus.cc:2781] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:37:58.667057 16605 raft_consensus.cc:686] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 LEADER]: Becoming Leader. State: Replica: 9310e650d22e4d3889b6186f9f3a2247, State: Running, Role: LEADER
I20260808 10:37:58.667342 16605 consensus_queue.cc:237] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:37:58.670686 16396 catalog_manager.cc:5495] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9310e650d22e4d3889b6186f9f3a2247 (127.30.146.129). New cstate: current_term: 1 leader_uuid: "9310e650d22e4d3889b6186f9f3a2247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } health_report { overall_health: HEALTHY } } }
I20260808 10:37:58.713985 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.709416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.30.146.129:46423)
0808 10:37:58.709874 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 10:37:58.709886 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:37:58.709905 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:37:58.709998 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:37:58.710001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:37:58.710074 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:37:58.710224 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:37:58.710462 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.711361 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.712188 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:37:58.712558 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:37:58.712674 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:37:58.712921 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:37:58.713544 (+ 623us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:37:58.713661 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:37:58.713737 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 10:37:58.713807 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":229,"threads_started":1}
I20260808 10:37:58.718647 16697 raft_consensus.cc:1261] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356 [term 1 FOLLOWER]: Refusing update from remote peer 9310e650d22e4d3889b6186f9f3a2247: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:58.718797 16837 raft_consensus.cc:1261] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4 [term 1 FOLLOWER]: Refusing update from remote peer 9310e650d22e4d3889b6186f9f3a2247: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:37:58.719383 16605 consensus_queue.cc:1035] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:58.719580 16911 consensus_queue.cc:1035] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:37:58.730639 16918 mvcc.cc:205] Tried to move back new op lower bound from 7316215720824025088 to 7316215720624189440. Current Snapshot: MvccSnapshot[applied={T|T < 7316215720824025088}]
I20260808 10:37:58.738925 16397 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:37:58.739140 16397 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:37:58.741817 16397 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Sending DeleteTablet for 3 replicas of tablet db92504b3bfe4b37a2c20a892a289df7
I20260808 10:37:58.742640 16675 tablet_service.cc:1514] Processing DeleteTablet for tablet db92504b3bfe4b37a2c20a892a289df7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:58 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:47804
I20260808 10:37:58.742659 16816 tablet_service.cc:1514] Processing DeleteTablet for tablet db92504b3bfe4b37a2c20a892a289df7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:37:58 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:47344
I20260808 10:37:58.742837 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 16438
I20260808 10:37:58.745222 16926 tablet_replica.cc:330] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: stopping tablet replica
I20260808 10:37:58.745618 16926 raft_consensus.cc:2229] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:58.745977 16926 raft_consensus.cc:2258] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:58.746487 16928 tablet_replica.cc:330] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: stopping tablet replica
I20260808 10:37:58.746915 16928 raft_consensus.cc:2229] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:37:58.747218 16928 raft_consensus.cc:2258] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:37:58.748973 16928 ts_tablet_manager.cc:1905] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:58.748973 16926 ts_tablet_manager.cc:1905] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:37:58.751989 16926 ts_tablet_manager.cc:1918] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:58.751989 16928 ts_tablet_manager.cc:1918] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:37:58.752125 16928 log.cc:1196] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/db92504b3bfe4b37a2c20a892a289df7
I20260808 10:37:58.752125 16926 log.cc:1196] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/db92504b3bfe4b37a2c20a892a289df7
I20260808 10:37:58.752523 16926 ts_tablet_manager.cc:1939] T db92504b3bfe4b37a2c20a892a289df7 P e02f043225a1467c98355320ed4ce356: Deleting consensus metadata
I20260808 10:37:58.752574 16928 ts_tablet_manager.cc:1939] T db92504b3bfe4b37a2c20a892a289df7 P 662378f8eb92444f9ed9d9a6b87638b4: Deleting consensus metadata
W20260808 10:37:58.753031 16380 connection.cc:535] client connection to 127.30.146.129:46423 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 10:37:58.753252 16380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:37:58.753472 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 16600
I20260808 10:37:58.757303 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.756925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:46423 (local address 127.0.0.1:54306)
0808 10:37:58.757102 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:58.757207 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:46423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 10:37:58.757527 16383 catalog_manager.cc:4841] TS 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111): tablet db92504b3bfe4b37a2c20a892a289df7 (table test-table [id=226f315ca4f849c3a6cd4ad0a51c915c]) successfully deleted
W20260808 10:37:58.757686 16380 catalog_manager.cc:4584] TS 9310e650d22e4d3889b6186f9f3a2247 (127.30.146.129:46423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet db92504b3bfe4b37a2c20a892a289df7: Network error: Client connection negotiation failed: client connection to 127.30.146.129:46423: connect: Connection refused (error 111)
I20260808 10:37:58.761562 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 16748
I20260808 10:37:58.763051 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:37:58.761869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:38867 (local address 127.0.0.1:47810)
0808 10:37:58.762848 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0808 10:37:58.762932 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.130:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":865}
I20260808 10:37:58.770116 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 16362
I20260808 10:38:18.778563 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33317
--webserver_interface=127.30.146.190
--webserver_port=42431
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:33317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:18.895447 16930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:18.895799 16930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:18.895880 16930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:18.899991 16930 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:38:18.900086 16930 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:38:18.900112 16930 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:38:18.900135 16930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:18.900175 16930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:18.900216 16930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:18.900255 16930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:18.900291 16930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:18.905867 16930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:33317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=42431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:18.907117 16930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:18.908259 16930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:18.918010 16935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:18.918221 16938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:18.918087 16930 server_base.cc:1034] running on GCE node
W20260808 10:38:18.918102 16936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:18.919023 16930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:18.920284 16930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:18.921452 16930 hybrid_clock.cc:648] HybridClock initialized: now 1786185498921430 us; error 35 us; skew 500 ppm
Aug 08 10:38:18 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185498, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:18.925665 16930 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:38:18.927114 16930 webserver.cc:460] Webserver started at http://127.30.146.190:42431/ using document root <none> and password file <none>
I20260808 10:38:18.927948 16930 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:18.928094 16930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:18.933765 16930 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:38:18.936234 16945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:18.937764 16930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:38:18.937903 16930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "b6b79a18f3894a5486b416a4d8ea5641"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "f9a38043a24c097ad6bcbf4d1e800db3"
server_key_iv: "f8bca7e6c6c667eb849b3c4f9deaede9"
server_key_version: "encryptionkey@0"
I20260808 10:38:18.938475 16930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:18.950423 16930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:18.953533 16930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:18.953743 16930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:18.961615 16930 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:33317
I20260808 10:38:18.962046 17000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:33317 every 8 connection(s)
I20260808 10:38:18.962775 16930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:18.967607 17001 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:38:18.967628 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 16930
I20260808 10:38:18.968133 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:46423
--local_ip_for_outbound_sockets=127.30.146.129
--tserver_master_addrs=127.30.146.190:33317
--webserver_port=46075
--webserver_interface=127.30.146.129
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:38:18.969825 17001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Bootstrap starting.
I20260808 10:38:18.991061 17001 log.cc:824] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:19.003222 17001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:38:19.003706 17001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Bootstrap complete.
I20260808 10:38:19.008476 17001 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:38:19.008941 17001 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b79a18f3894a5486b416a4d8ea5641, State: Initialized, Role: FOLLOWER
I20260808 10:38:19.009550 17001 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:38:19.009716 17001 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:19.009810 17001 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:19.009936 17001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:38:19.012136 17001 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:38:19.012514 17001 leader_election.cc:302] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6b79a18f3894a5486b416a4d8ea5641; no voters:
I20260808 10:38:19.012907 17001 leader_election.cc:288] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 10:38:19.013019 17006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:38:19.013253 17006 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [term 3 LEADER]: Becoming Leader. State: Replica: b6b79a18f3894a5486b416a4d8ea5641, State: Running, Role: LEADER
I20260808 10:38:19.013586 17006 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } }
I20260808 10:38:19.014384 17001 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:19.014680 17008 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6b79a18f3894a5486b416a4d8ea5641. Latest consensus state: current_term: 3 leader_uuid: "b6b79a18f3894a5486b416a4d8ea5641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } } }
I20260808 10:38:19.014814 17008 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:19.015213 17007 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b6b79a18f3894a5486b416a4d8ea5641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b79a18f3894a5486b416a4d8ea5641" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33317 } } }
I20260808 10:38:19.015342 17007 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:19.016135 17010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:19.021975 17010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:19.022758 17010 catalog_manager.cc:1259] Loaded cluster ID: f2be7bf1d1f64167bf0235ffb3f2f36f
I20260808 10:38:19.022826 17010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:19.025671 17010 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:19.026523 17010 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Loaded TSK: 0
I20260808 10:38:19.026629 17010 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b6b79a18f3894a5486b416a4d8ea5641: Loaded TSK: 1
F20260808 10:38:19.027534 17010 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f024be899ad google::LogMessage::Fail() at ??:0
@ 0x7f024be8c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f024be894de google::LogMessage::Flush() at ??:0
@ 0x7f024be8aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f024eb4d18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f024eb45f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f02567f27bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f02567cc99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f02567cd178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f025680edf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f025682d82b std::function<>::operator()() at ??:0
@ 0x7f02567ccb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f02567cdf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f02567ca579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f025680e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f024e0db3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f024e0dbc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f024e0dd3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f024e0cc512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f024f7106db start_thread at ??:0
@ 0x7f024a98761f clone at ??:0
W20260808 10:38:19.113734 17003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:19.114147 17003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:19.114255 17003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:19.118777 17003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:19.118897 17003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:19.119011 17003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:38:19.125286 17003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:46423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=46075
--tserver_master_addrs=127.30.146.190:33317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:19.126375 17003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:19.127390 17003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:19.138149 17052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:19.138439 17003 server_base.cc:1034] running on GCE node
W20260808 10:38:19.138602 17051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:19.138701 17055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:19.139123 17003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:19.139832 17003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:19.141033 17003 hybrid_clock.cc:648] HybridClock initialized: now 1786185499141002 us; error 51 us; skew 500 ppm
Aug 08 10:38:19 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185499, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:19.145104 17003 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:38:19.146237 17003 webserver.cc:460] Webserver started at http://127.30.146.129:46075/ using document root <none> and password file <none>
I20260808 10:38:19.146777 17003 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:19.146840 17003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:19.151163 17003 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 10:38:19.153512 17065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:19.155158 17003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:38:19.155346 17003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "9310e650d22e4d3889b6186f9f3a2247"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "d972e00e2f48a0172d6fa810d64f651c"
server_key_iv: "f7b57aa37a074af82046d4b6fecca154"
server_key_version: "encryptionkey@0"
I20260808 10:38:19.156160 17003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:19.168820 17003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:19.172621 17003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:19.172850 17003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:19.173527 17003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:19.174696 17072 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:38:19.180843 17003 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:38:19.180924 17003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 10:38:19.181018 17003 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:38:19.182765 17003 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:38:19.182838 17003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 10:38:19.183153 17072 tablet_bootstrap.cc:492] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Bootstrap starting.
I20260808 10:38:19.193955 17003 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:46423
I20260808 10:38:19.194054 17186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:46423 every 8 connection(s)
I20260808 10:38:19.194969 17003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:38:19.196444 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 17003
I20260808 10:38:19.197000 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:38867
--local_ip_for_outbound_sockets=127.30.146.130
--tserver_master_addrs=127.30.146.190:33317
--webserver_port=45757
--webserver_interface=127.30.146.130
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:38:19.209816 17072 log.cc:824] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:19.216977 17072 tablet_bootstrap.cc:492] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:38:19.217422 17072 tablet_bootstrap.cc:492] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Bootstrap complete.
I20260808 10:38:19.218837 17072 ts_tablet_manager.cc:1397] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Time spent bootstrapping tablet: real 0.036s user 0.020s sys 0.011s
I20260808 10:38:19.221315 17072 raft_consensus.cc:348] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:38:19.221760 17072 raft_consensus.cc:729] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9310e650d22e4d3889b6186f9f3a2247, State: Initialized, Role: FOLLOWER
I20260808 10:38:19.222254 17072 consensus_queue.cc:260] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:38:19.223217 17072 ts_tablet_manager.cc:1428] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260808 10:38:19.320161 17190 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:38:19.197274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:35355)
0808 10:38:19.199406 (+ 2132us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:19.199462 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:19.200325 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:19.319215 (+118890us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1887,"thread_start_us":1567,"threads_started":1}
W20260808 10:38:19.320822 17187 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:33317 (0 consecutive failures): Network error: Failed to ping master at 127.30.146.190:33317: Client connection negotiation failed: client connection to 127.30.146.190:33317: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:38:19.321651 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:19.321394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:50959)
0808 10:38:19.321520 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:19.321596 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:38:19.322477 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:19.322225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:59397)
0808 10:38:19.322346 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:19.322407 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 10:38:19.322688 17187 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:38:19.345944 17189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:19.346249 17189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:19.346297 17189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:19.350476 17189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:19.350551 17189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:19.350622 17189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:38:19.356258 17189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:38867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=45757
--tserver_master_addrs=127.30.146.190:33317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:19.357416 17189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:19.358470 17189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:19.369365 17197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:19.369365 17198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:19.369464 17189 server_base.cc:1034] running on GCE node
W20260808 10:38:19.369369 17200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:19.370172 17189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:19.370913 17189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:19.372072 17189 hybrid_clock.cc:648] HybridClock initialized: now 1786185499372042 us; error 47 us; skew 500 ppm
Aug 08 10:38:19 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185499, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:19.375031 17189 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:38:19.375998 17189 webserver.cc:460] Webserver started at http://127.30.146.130:45757/ using document root <none> and password file <none>
I20260808 10:38:19.376528 17189 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:19.376590 17189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:19.380678 17189 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:38:19.382979 17207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:19.384341 17189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:38:19.384480 17189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "e02f043225a1467c98355320ed4ce356"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "f5673d70d7ffab605ec5b1ad92480006"
server_key_iv: "2500cc6151dee5de61e24cf751fcb4bd"
server_key_version: "encryptionkey@0"
I20260808 10:38:19.385133 17189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:19.413399 17189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:19.416705 17189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:19.416916 17189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:19.417615 17189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:19.418583 17189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:19.418650 17189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:19.418730 17189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:19.418769 17189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:19.428263 17189 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:38867
I20260808 10:38:19.428318 17329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:38867 every 8 connection(s)
I20260808 10:38:19.429355 17189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:38:19.431649 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 17189
I20260808 10:38:19.432291 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:39111
--local_ip_for_outbound_sockets=127.30.146.131
--tserver_master_addrs=127.30.146.190:33317
--webserver_port=42747
--webserver_interface=127.30.146.131
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:38:19.433702 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:19.431586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:38175)
0808 10:38:19.432522 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:19.432675 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":149,"threads_started":1}
W20260808 10:38:19.434334 17330 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:33317 (0 consecutive failures): Network error: Failed to ping master at 127.30.146.190:33317: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
I20260808 10:38:19.435103 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:19.434809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:33609)
0808 10:38:19.434974 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:19.435039 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 10:38:19.435806 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:19.435584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:59971)
0808 10:38:19.435699 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:19.435753 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 10:38:19.436031 17330 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:38:19.549314 17333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:19.549650 17333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:19.549731 17333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:19.554344 17333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:19.554440 17333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:19.554544 17333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:38:19.560542 17333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:39111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=42747
--tserver_master_addrs=127.30.146.190:33317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:19.561815 17333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:19.562996 17333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:19.573465 17342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:19.573465 17340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:19.573460 17338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:19.573832 17333 server_base.cc:1034] running on GCE node
I20260808 10:38:19.574294 17333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:19.574928 17333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:19.576139 17333 hybrid_clock.cc:648] HybridClock initialized: now 1786185499576111 us; error 51 us; skew 500 ppm
Aug 08 10:38:19 dist-test-slave-9gcw krb5kdc[15769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185499, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:19.579131 17333 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:38:19.580078 17333 webserver.cc:460] Webserver started at http://127.30.146.131:42747/ using document root <none> and password file <none>
I20260808 10:38:19.580642 17333 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:19.580790 17333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:19.584810 17333 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:38:19.586988 17349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:19.588266 17333 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:38:19.588393 17333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "662378f8eb92444f9ed9d9a6b87638b4"
format_stamp: "Formatted at 2026-08-08 10:37:36 on dist-test-slave-9gcw"
server_key: "4bc2bfb82cdfa88a194f284d6dca2dd1"
server_key_iv: "2e530e075049d5441e5993d282477e9e"
server_key_version: "encryptionkey@0"
I20260808 10:38:19.588994 17333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:19.611454 17333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:19.614889 17333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:19.615108 17333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:19.615763 17333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:19.616626 17333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:19.616695 17333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:19.616778 17333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:19.616825 17333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:19.627022 17333 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:39111
I20260808 10:38:19.627066 17472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:39111 every 8 connection(s)
I20260808 10:38:19.628077 17333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:38:19.631208 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 17333
I20260808 10:38:19.631989 17475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:19.630250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:39489)
0808 10:38:19.630943 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:19.631057 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":118,"threads_started":1}
W20260808 10:38:19.632437 17473 heartbeater.cc:644] Failed to heartbeat to 127.30.146.190:33317 (0 consecutive failures): Network error: Failed to ping master at 127.30.146.190:33317: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
I20260808 10:38:19.633008 17475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:19.632809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:51367)
0808 10:38:19.632907 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:19.632953 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 10:38:19.633587 17475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:19.633408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:48677)
0808 10:38:19.633500 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:19.633542 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 10:38:19.633749 17473 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:38:20.323832 17477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:20.323320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:47119)
0808 10:38:20.323595 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:20.323661 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":81,"threads_started":1}
I20260808 10:38:20.382840 17478 raft_consensus.cc:482] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:20.383091 17478 raft_consensus.cc:504] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:38:20.384259 17478 leader_election.cc:288] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111), e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867)
Aug 08 10:38:20 dist-test-slave-9gcw krb5kdc[15769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185499, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.130@KRBTEST.COM
Aug 08 10:38:20 dist-test-slave-9gcw krb5kdc[15769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185499, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.131@KRBTEST.COM
I20260808 10:38:20.398897 17479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:20.384525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:38867 (local address 127.30.146.129:34301)
0808 10:38:20.385103 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:20.385121 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:20.385270 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:20.386772 (+ 1502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:20.386778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:20.387346 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:20.387818 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:20.387832 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:20.389238 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:20.389242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:20.390173 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:20.390183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:20.390883 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:20.390887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:20.391101 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:20.391814 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:20.391826 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:20.393637 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:20.396069 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:20.396075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:20.396089 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:20.396445 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:20.396813 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:20.396818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:20.396821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:20.397005 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:20.397841 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:20.397846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:20.398150 (+ 304us) client_negotiation.cc:769] Sending connection context
0808 10:38:20.398363 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 10:38:20.398670 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":288,"spinlock_wait_cycles":43904,"thread_start_us":73,"threads_started":1}
I20260808 10:38:20.399084 17477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:20.384525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:39111 (local address 127.30.146.129:41879)
0808 10:38:20.385167 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:20.385183 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:20.385315 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:20.386772 (+ 1457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:20.386778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:20.387347 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:20.387817 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:20.387832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:20.389265 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:20.389268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:20.390170 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:20.390180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:20.390883 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:20.390887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:20.391100 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:20.391704 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:20.391733 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:20.394510 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:20.397048 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:20.397050 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:20.397052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:20.397351 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:20.397697 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:20.397700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:20.397702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:20.397807 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:20.398508 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:20.398510 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:20.398602 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 10:38:20.398703 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 10:38:20.398831 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":62,"spinlock_wait_cycles":8320}
I20260808 10:38:20.399281 17481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:20.384814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:34301 (local address 127.30.146.130:38867)
0808 10:38:20.385307 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:20.385325 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:20.385371 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:38:20.386039 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:20.386045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:20.386400 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:20.386720 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:20.388005 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:20.389071 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:20.390297 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:20.390748 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:20.390920 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:20.393926 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:20.393931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:20.393942 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:20.393972 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:20.395943 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:20.396561 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:20.396565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:20.396569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:20.396682 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:20.397113 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:20.397119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:20.397123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:20.397680 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:20.397861 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:20.398482 (+ 621us) server_negotiation.cc:299] Negotiation successful
0808 10:38:20.398828 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":107,"threads_started":1}
I20260808 10:38:20.399343 17480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:20.384787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:41879 (local address 127.30.146.131:39111)
0808 10:38:20.385134 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:20.385171 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:20.385295 (+ 124us) server_negotiation.cc:407] Connection header received
0808 10:38:20.386037 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:20.386043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:20.386400 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:20.386727 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:20.388000 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:20.389070 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:20.390303 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:20.390749 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:20.390924 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:20.394773 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:20.394779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:20.394789 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:20.394818 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:20.396931 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:20.397478 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:20.397483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:20.397487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:20.397595 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:20.397916 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:20.397919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:20.397921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:20.398398 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:20.398516 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:20.398871 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 10:38:20.399142 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":107,"threads_started":1}
W20260808 10:38:20.401597 17069 leader_election.cc:341] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111): Not found: Tablet not found: db92504b3bfe4b37a2c20a892a289df7
W20260808 10:38:20.401768 17067 leader_election.cc:341] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867): Not found: Tablet not found: db92504b3bfe4b37a2c20a892a289df7
I20260808 10:38:20.401880 17067 leader_election.cc:302] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9310e650d22e4d3889b6186f9f3a2247; no voters: 662378f8eb92444f9ed9d9a6b87638b4, e02f043225a1467c98355320ed4ce356
I20260808 10:38:20.402158 17478 raft_consensus.cc:2726] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:38:20.437374 17482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:20.436834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:48735)
0808 10:38:20.437195 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:20.437262 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":131,"threads_started":1}
I20260808 10:38:20.634943 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:20.634412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:47481)
0808 10:38:20.634777 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:20.634838 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":96,"threads_started":1}
I20260808 10:38:21.325117 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:21.324645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:57223)
0808 10:38:21.324951 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:21.325015 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":81,"threads_started":1}
I20260808 10:38:21.438692 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:21.438213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:50249)
0808 10:38:21.438492 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:21.438555 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":78,"threads_started":1}
I20260808 10:38:21.636387 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:21.635879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:42099)
0808 10:38:21.636197 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:21.636270 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":101,"threads_started":1}
I20260808 10:38:22.326436 17487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:22.326013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:51657)
0808 10:38:22.326277 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:22.326336 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260808 10:38:22.439867 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:22.439448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:60389)
0808 10:38:22.439713 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:22.439771 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
I20260808 10:38:22.449430 17490 raft_consensus.cc:482] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:22.449594 17490 raft_consensus.cc:504] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:38:22.450059 17490 leader_election.cc:288] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111), e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867)
W20260808 10:38:22.450932 17069 leader_election.cc:341] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111): Not found: Tablet not found: db92504b3bfe4b37a2c20a892a289df7
W20260808 10:38:22.451077 17067 leader_election.cc:341] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867): Not found: Tablet not found: db92504b3bfe4b37a2c20a892a289df7
I20260808 10:38:22.451156 17067 leader_election.cc:302] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9310e650d22e4d3889b6186f9f3a2247; no voters: 662378f8eb92444f9ed9d9a6b87638b4, e02f043225a1467c98355320ed4ce356
I20260808 10:38:22.451337 17490 raft_consensus.cc:2726] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:38:22.637900 17491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:22.637394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:50109)
0808 10:38:22.637739 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:22.637802 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":86,"threads_started":1}
I20260808 10:38:23.327847 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:23.327286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:59843)
0808 10:38:23.327645 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:23.327718 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":124,"threads_started":1}
I20260808 10:38:23.441222 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:23.440727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:51659)
0808 10:38:23.440987 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:23.441046 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
I20260808 10:38:23.639161 17494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:23.638708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:47023)
0808 10:38:23.638988 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:23.639052 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":84,"threads_started":1}
I20260808 10:38:24.329406 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:24.328802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:45661)
0808 10:38:24.329137 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:24.329302 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":80,"threads_started":1}
I20260808 10:38:24.442437 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:24.442010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:52253)
0808 10:38:24.442279 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:24.442340 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260808 10:38:24.640702 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:24.640139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:38057)
0808 10:38:24.640465 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:24.640531 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":109,"threads_started":1}
I20260808 10:38:25.330896 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:25.330424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:34963)
0808 10:38:25.330720 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:25.330791 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":90,"threads_started":1}
I20260808 10:38:25.355826 17499 raft_consensus.cc:482] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:25.355973 17499 raft_consensus.cc:504] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:38:25.356529 17499 leader_election.cc:288] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111), e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867)
W20260808 10:38:25.357534 17069 leader_election.cc:341] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111): Not found: Tablet not found: db92504b3bfe4b37a2c20a892a289df7
W20260808 10:38:25.357651 17067 leader_election.cc:341] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867): Not found: Tablet not found: db92504b3bfe4b37a2c20a892a289df7
I20260808 10:38:25.357779 17067 leader_election.cc:302] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9310e650d22e4d3889b6186f9f3a2247; no voters: 662378f8eb92444f9ed9d9a6b87638b4, e02f043225a1467c98355320ed4ce356
I20260808 10:38:25.357963 17499 raft_consensus.cc:2726] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:38:25.443856 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:25.443340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:56669)
0808 10:38:25.443661 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:25.443741 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":86,"threads_started":1}
I20260808 10:38:25.642192 17501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:25.641620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:42009)
0808 10:38:25.641979 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:25.642062 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":112,"threads_started":1}
I20260808 10:38:26.332235 17502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:26.331782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:59931)
0808 10:38:26.332072 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:26.332133 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":80,"threads_started":1}
I20260808 10:38:26.445286 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:26.444727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:38715)
0808 10:38:26.445039 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:26.445100 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":81,"threads_started":1}
I20260808 10:38:26.643652 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:26.643091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:47679)
0808 10:38:26.643481 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:26.643547 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":94,"threads_started":1}
I20260808 10:38:27.198025 17505 raft_consensus.cc:482] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:27.198143 17505 raft_consensus.cc:504] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:38:27.198515 17505 leader_election.cc:288] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111), e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867)
W20260808 10:38:27.199360 17067 leader_election.cc:341] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867): Not found: Tablet not found: db92504b3bfe4b37a2c20a892a289df7
W20260808 10:38:27.199469 17069 leader_election.cc:341] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111): Not found: Tablet not found: db92504b3bfe4b37a2c20a892a289df7
I20260808 10:38:27.199534 17069 leader_election.cc:302] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9310e650d22e4d3889b6186f9f3a2247; no voters: 662378f8eb92444f9ed9d9a6b87638b4, e02f043225a1467c98355320ed4ce356
I20260808 10:38:27.199666 17505 raft_consensus.cc:2726] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:38:27.333611 17506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:27.333124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:42989)
0808 10:38:27.333451 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:27.333510 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":117,"threads_started":1}
I20260808 10:38:27.446591 17507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:27.446051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:40393)
0808 10:38:27.446402 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:27.446484 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":96,"threads_started":1}
I20260808 10:38:27.644991 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:27.644517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:39261)
0808 10:38:27.644825 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:27.644889 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":86,"threads_started":1}
I20260808 10:38:28.335122 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:28.334572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:44563)
0808 10:38:28.334928 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:28.335009 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":86,"threads_started":1}
I20260808 10:38:28.447923 17510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:28.447466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:44671)
0808 10:38:28.447759 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:28.447822 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":86,"threads_started":1}
I20260808 10:38:28.646293 17511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:28.645779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:59351)
0808 10:38:28.646120 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:28.646187 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
I20260808 10:38:29.336483 17512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:29.336066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:37487)
0808 10:38:29.336327 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:29.336383 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
I20260808 10:38:29.449237 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:29.448729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:50535)
0808 10:38:29.449004 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:29.449064 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":78,"threads_started":1}
I20260808 10:38:29.647740 17514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:29.647226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:33283)
0808 10:38:29.647547 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:29.647618 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":85,"threads_started":1}
I20260808 10:38:30.337996 17515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:30.337390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:53849)
0808 10:38:30.337807 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:30.337881 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":91,"threads_started":1}
I20260808 10:38:30.450685 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:30.450103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:50713)
0808 10:38:30.450500 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:30.450574 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":109,"threads_started":1}
I20260808 10:38:30.649237 17519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:30.648702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:40559)
0808 10:38:30.649001 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:30.649064 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":84,"threads_started":1}
I20260808 10:38:31.339382 17520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:31.338897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:59269)
0808 10:38:31.339207 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:31.339274 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
I20260808 10:38:31.451988 17521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:31.451551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:34677)
0808 10:38:31.451823 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:31.451885 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":78,"threads_started":1}
I20260808 10:38:31.650769 17522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:31.650114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:50327)
0808 10:38:31.650497 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:31.650624 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":148,"threads_started":1}
I20260808 10:38:32.340850 17523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:32.340309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:39771)
0808 10:38:32.340679 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:32.340747 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":173,"threads_started":1}
I20260808 10:38:32.453522 17524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:32.452972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:44087)
0808 10:38:32.453369 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:32.453423 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":153,"threads_started":1}
I20260808 10:38:32.652366 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:32.651745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:60635)
0808 10:38:32.652070 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:32.652132 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":142,"threads_started":1}
I20260808 10:38:33.342219 17527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:33.341765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:53699)
0808 10:38:33.342064 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:33.342118 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":105,"threads_started":1}
I20260808 10:38:33.454838 17528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:33.454410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:43535)
0808 10:38:33.454667 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:33.454732 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":77,"threads_started":1}
I20260808 10:38:33.653935 17529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:33.653430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.131:34261)
0808 10:38:33.653758 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:33.653825 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":93,"threads_started":1}
I20260808 10:38:33.989933 17530 raft_consensus.cc:482] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:33.990070 17530 raft_consensus.cc:504] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662378f8eb92444f9ed9d9a6b87638b4" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 39111 } } peers { permanent_uuid: "e02f043225a1467c98355320ed4ce356" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 38867 } } peers { permanent_uuid: "9310e650d22e4d3889b6186f9f3a2247" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 46423 } }
I20260808 10:38:33.990476 17530 leader_election.cc:288] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111), e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867)
W20260808 10:38:33.991487 17069 leader_election.cc:341] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 662378f8eb92444f9ed9d9a6b87638b4 (127.30.146.131:39111): Not found: Tablet not found: db92504b3bfe4b37a2c20a892a289df7
W20260808 10:38:33.991755 17067 leader_election.cc:341] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e02f043225a1467c98355320ed4ce356 (127.30.146.130:38867): Not found: Tablet not found: db92504b3bfe4b37a2c20a892a289df7
I20260808 10:38:33.991839 17067 leader_election.cc:302] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9310e650d22e4d3889b6186f9f3a2247; no voters: 662378f8eb92444f9ed9d9a6b87638b4, e02f043225a1467c98355320ed4ce356
I20260808 10:38:33.992040 17530 raft_consensus.cc:2726] T db92504b3bfe4b37a2c20a892a289df7 P 9310e650d22e4d3889b6186f9f3a2247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:38:34.343786 17531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:34.343279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.129:56797)
0808 10:38:34.343598 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:34.343666 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":90,"threads_started":1}
I20260808 10:38:34.456305 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:34.455820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33317 (local address 127.30.146.130:46381)
0808 10:38:34.456128 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:34.456197 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.190:33317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":79,"threads_started":1}
I20260808 10:38:34.633713 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 17003
I20260808 10:38:34.643167 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 17189
I20260808 10:38:34.651605 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 17333
2026-08-08T10:38:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59895 ms)
[----------] 30 tests from SecurityITest (257956 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:34 dist-test-slave-9gcw krb5kdc[17537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:34 dist-test-slave-9gcw krb5kdc[17537](info): set up 2 sockets
Aug 08 10:38:34 dist-test-slave-9gcw krb5kdc[17537](info): commencing operation
krb5kdc: starting...
W20260808 10:38:35.822283 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:35 dist-test-slave-9gcw krb5kdc[17537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:38:35.956146 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:40047
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:40047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:36.082870 17553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:36.083199 17553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:36.083279 17553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:36.087857 17553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:36.087961 17553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:36.088004 17553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:36.088049 17553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:36.088088 17553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:36.094192 17553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:40047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:40047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:36.095533 17553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:36.096725 17553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:36.106472 17559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:36.106447 17562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:36.106503 17553 server_base.cc:1034] running on GCE node
W20260808 10:38:36.106451 17560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:36.107403 17553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:36.108661 17553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:36.109933 17553 hybrid_clock.cc:648] HybridClock initialized: now 1786185516109879 us; error 42 us; skew 500 ppm
Aug 08 10:38:36 dist-test-slave-9gcw krb5kdc[17537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:36.112735 17553 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:38:36.113768 17553 webserver.cc:460] Webserver started at http://127.30.146.190:38521/ using document root <none> and password file <none>
I20260808 10:38:36.114349 17553 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:36.114444 17553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:36.114656 17553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:36.116753 17553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "e688b7f983584aa8a0cd229e91d0f392"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "d197b63e5b8839c2e82f9a1b6c869273"
server_key_iv: "637aa06bd39b5e593fa5ac535f71b533"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.117398 17553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "e688b7f983584aa8a0cd229e91d0f392"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "d197b63e5b8839c2e82f9a1b6c869273"
server_key_iv: "637aa06bd39b5e593fa5ac535f71b533"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.121244 17553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:38:36.123668 17569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:36.124977 17553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:38:36.125079 17553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "e688b7f983584aa8a0cd229e91d0f392"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "d197b63e5b8839c2e82f9a1b6c869273"
server_key_iv: "637aa06bd39b5e593fa5ac535f71b533"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.125252 17553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:36.185495 17553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:36.189886 17553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:36.190184 17553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:36.199697 17553 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:40047
I20260808 10:38:36.199725 17628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:40047 every 8 connection(s)
I20260808 10:38:36.200901 17553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:36.203733 17629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:36.204818 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 17553
I20260808 10:38:36.204974 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:36.205364 31306 external_mini_cluster.cc:1428] Setting key fbbd9c1471a213e8c205b03146acb859
I20260808 10:38:36.210424 17629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392: Bootstrap starting.
I20260808 10:38:36.213253 17629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:36.214298 17629 log.cc:824] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:38:36 dist-test-slave-9gcw krb5kdc[17537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:36.216854 17629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392: No bootstrap required, opened a new log
I20260808 10:38:36.220777 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:36.206979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58980 (local address 127.30.146.190:40047)
0808 10:38:36.207410 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:36.207428 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:36.207475 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:38:36.208109 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:36.208125 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:36.208462 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:36.208741 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:36.209745 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.210770 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.211936 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.212393 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.212504 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:36.215598 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:36.215604 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:36.215619 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:36.215650 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:36.217633 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:36.218056 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:36.218062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:36.218068 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:36.218214 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:36.218470 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:36.218473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:36.218475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:36.218987 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:36.219224 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:36.219396 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 10:38:36.219726 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":141,"threads_started":1}
I20260808 10:38:36.221629 17629 raft_consensus.cc:348] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e688b7f983584aa8a0cd229e91d0f392" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40047 } }
I20260808 10:38:36.221879 17629 raft_consensus.cc:374] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:36.221939 17629 raft_consensus.cc:729] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e688b7f983584aa8a0cd229e91d0f392, State: Initialized, Role: FOLLOWER
I20260808 10:38:36.222393 17629 consensus_queue.cc:260] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e688b7f983584aa8a0cd229e91d0f392" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40047 } }
I20260808 10:38:36.222546 17629 raft_consensus.cc:388] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:36.222612 17629 raft_consensus.cc:482] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:36.222716 17629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:36.223628 17629 raft_consensus.cc:504] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e688b7f983584aa8a0cd229e91d0f392" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40047 } }
I20260808 10:38:36.223976 17629 leader_election.cc:302] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e688b7f983584aa8a0cd229e91d0f392; no voters:
I20260808 10:38:36.224290 17629 leader_election.cc:288] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:36.224417 17635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:36.224650 17635 raft_consensus.cc:686] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [term 1 LEADER]: Becoming Leader. State: Replica: e688b7f983584aa8a0cd229e91d0f392, State: Running, Role: LEADER
I20260808 10:38:36.224926 17635 consensus_queue.cc:237] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e688b7f983584aa8a0cd229e91d0f392" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40047 } }
I20260808 10:38:36.225666 17629 sys_catalog.cc:564] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:36.226687 17636 sys_catalog.cc:455] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e688b7f983584aa8a0cd229e91d0f392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e688b7f983584aa8a0cd229e91d0f392" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40047 } } }
I20260808 10:38:36.226897 17636 sys_catalog.cc:458] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:36.227164 17637 sys_catalog.cc:455] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e688b7f983584aa8a0cd229e91d0f392. Latest consensus state: current_term: 1 leader_uuid: "e688b7f983584aa8a0cd229e91d0f392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e688b7f983584aa8a0cd229e91d0f392" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40047 } } }
I20260808 10:38:36.227267 17637 sys_catalog.cc:458] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:36.227329 17644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:36.230589 17644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:36.236109 17644 catalog_manager.cc:1347] Generated new cluster ID: 1f75413e84f541b4a1ee08256fe93578
I20260808 10:38:36.236176 17644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:36.243263 17644 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:36.244108 17644 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:36.249832 17644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392: Generated new TSK 0
I20260808 10:38:36.250461 17644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:38:36.284299 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:40047
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:36.404752 17658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:36.405066 17658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:36.405198 17658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:36.409261 17658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:36.409348 17658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:36.409444 17658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:38:36.414849 17658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:40047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:36.415973 17658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:36.417034 17658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:36.427063 17663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:36.427026 17666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:36.427264 17658 server_base.cc:1034] running on GCE node
W20260808 10:38:36.427433 17664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:36.427861 17658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:36.428596 17658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:36.429795 17658 hybrid_clock.cc:648] HybridClock initialized: now 1786185516429767 us; error 57 us; skew 500 ppm
Aug 08 10:38:36 dist-test-slave-9gcw krb5kdc[17537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:36.432832 17658 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:38:36.433871 17658 webserver.cc:460] Webserver started at http://127.30.146.129:44979/ using document root <none> and password file <none>
I20260808 10:38:36.434453 17658 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:36.434520 17658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:36.434755 17658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:36.436866 17658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "b34ba08087e0413dbc033ed665d2c240"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "441a122cf9ae20a2628e2379362cd39a"
server_key_iv: "00be8991559fc2973a16a72e0cbbb762"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.437502 17658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "b34ba08087e0413dbc033ed665d2c240"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "441a122cf9ae20a2628e2379362cd39a"
server_key_iv: "00be8991559fc2973a16a72e0cbbb762"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.441404 17658 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:38:36.443883 17673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:36.445132 17658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:38:36.445320 17658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "b34ba08087e0413dbc033ed665d2c240"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "441a122cf9ae20a2628e2379362cd39a"
server_key_iv: "00be8991559fc2973a16a72e0cbbb762"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.445426 17658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:36.470356 17658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:36.473897 17658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:36.474116 17658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:36.474815 17658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:36.475698 17658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:36.475744 17658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:36.475814 17658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:36.475845 17658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:36.485828 17658 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:36549
I20260808 10:38:36.485868 17797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:36549 every 8 connection(s)
I20260808 10:38:36.486795 17658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:38:36.492555 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 17658
I20260808 10:38:36.492709 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:38:36.493088 31306 external_mini_cluster.cc:1428] Setting key 6e303806d3840a8848a409531c06f9b0
Aug 08 10:38:36 dist-test-slave-9gcw krb5kdc[17537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:36.502810 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:36.488696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:56131 (local address 127.30.146.190:40047)
0808 10:38:36.488858 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:36.488862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:36.489829 (+ 967us) server_negotiation.cc:407] Connection header received
0808 10:38:36.491284 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:36.491287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:36.491350 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:36.491432 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:36.492766 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.493588 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.495379 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.495757 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.495889 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:36.498988 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:36.498992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:36.498995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:36.499030 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:36.500601 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:36.501201 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:36.501205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:36.501206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:36.501282 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:36.501661 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:36.501666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:36.501667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:36.501863 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:36.501975 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:36.502508 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 10:38:36.502639 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 10:38:36.503726 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:36.489088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:40047 (local address 127.30.146.129:56131)
0808 10:38:36.489618 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:36.489716 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:36.491067 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:36.491606 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:36.491613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:36.491981 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:36.492553 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:36.492568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.493811 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.493816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:36.495218 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:36.495229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.495943 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.495947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:36.496101 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:36.496858 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:36.496876 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:36.498785 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:36.500741 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:36.500748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:36.500764 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:36.501050 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:36.501400 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:36.501403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:36.501406 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:36.501560 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:36.501991 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:36.502000 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:36.502265 (+ 265us) client_negotiation.cc:769] Sending connection context
0808 10:38:36.502467 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 10:38:36.502834 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:38:36.505062 17798 heartbeater.cc:347] Connected to a master server at 127.30.146.190:40047
I20260808 10:38:36.505486 17798 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:36.506036 17798 heartbeater.cc:511] Master 127.30.146.190:40047 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:38:36.507741 17587 ts_manager.cc:194] Registered new tserver with Master: b34ba08087e0413dbc033ed665d2c240 (127.30.146.129:36549)
I20260808 10:38:36.509483 17587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:56131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:38:36.525118 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:40047
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:36.654804 17803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:36.655110 17803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:36.655165 17803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:36.659407 17803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:36.659484 17803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:36.659550 17803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:38:36.665040 17803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:40047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:36.666177 17803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:36.667218 17803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:36.677492 17809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:36.677486 17812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:36.677510 17810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:36.677857 17803 server_base.cc:1034] running on GCE node
I20260808 10:38:36.678328 17803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:36.678999 17803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:36.680187 17803 hybrid_clock.cc:648] HybridClock initialized: now 1786185516680161 us; error 49 us; skew 500 ppm
Aug 08 10:38:36 dist-test-slave-9gcw krb5kdc[17537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:36.683266 17803 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:38:36.684242 17803 webserver.cc:460] Webserver started at http://127.30.146.130:35001/ using document root <none> and password file <none>
I20260808 10:38:36.684821 17803 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:36.684916 17803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:36.685220 17803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:36.687294 17803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "40fa985ef2bb423b82ae1f6caab8bb6e"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "5c613bd787b1a5605179b36752ea719f"
server_key_iv: "9e038606c1b7509ecf1859b7282b0e6a"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.687891 17803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "40fa985ef2bb423b82ae1f6caab8bb6e"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "5c613bd787b1a5605179b36752ea719f"
server_key_iv: "9e038606c1b7509ecf1859b7282b0e6a"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.691753 17803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 10:38:36.694299 17819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:36.695554 17803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:38:36.695693 17803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "40fa985ef2bb423b82ae1f6caab8bb6e"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "5c613bd787b1a5605179b36752ea719f"
server_key_iv: "9e038606c1b7509ecf1859b7282b0e6a"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.695807 17803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:36.717335 17803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:36.720336 17803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:36.720544 17803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:36.721207 17803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:36.722076 17803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:36.722148 17803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:36.722229 17803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:36.722280 17803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:36.732944 17803 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:41513
I20260808 10:38:36.733062 17944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:41513 every 8 connection(s)
I20260808 10:38:36.733959 17803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:38:36.743711 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 17803
I20260808 10:38:36.743898 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:38:36.744266 31306 external_mini_cluster.cc:1428] Setting key 764b11fdad9b8f4a7b53994d78c05bb5
Aug 08 10:38:36 dist-test-slave-9gcw krb5kdc[17537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:36.748718 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:36.736017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:43813 (local address 127.30.146.190:40047)
0808 10:38:36.736165 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:36.736168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:36.737267 (+ 1099us) server_negotiation.cc:407] Connection header received
0808 10:38:36.738823 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:36.738827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:36.738888 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:36.738974 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:36.740173 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.740726 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.741974 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.742217 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.742338 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:36.744982 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:36.744985 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:36.744987 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:36.745012 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:36.746598 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:36.747155 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:36.747160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:36.747162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:36.747247 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:36.747594 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:36.747598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:36.747600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:36.747793 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:36.747905 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:36.748406 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 10:38:36.748538 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:38:36.749730 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:36.736453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:40047 (local address 127.30.146.130:43813)
0808 10:38:36.736988 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:36.737032 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:36.738534 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:36.739141 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:36.739148 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:36.739518 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:36.740019 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:36.740033 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.740861 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.740865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:36.741842 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:36.741852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.742381 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.742385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:36.742572 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:36.743129 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:36.743145 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:36.744807 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:36.746761 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:36.746766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:36.746780 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:36.747046 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:36.747342 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:36.747345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:36.747347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:36.747494 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:36.747906 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:36.747914 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:36.748163 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 10:38:36.748364 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 10:38:36.748687 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":150,"threads_started":1}
I20260808 10:38:36.750954 17946 heartbeater.cc:347] Connected to a master server at 127.30.146.190:40047
I20260808 10:38:36.751225 17946 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:36.751766 17946 heartbeater.cc:511] Master 127.30.146.190:40047 requested a full tablet report, sending...
I20260808 10:38:36.752900 17587 ts_manager.cc:194] Registered new tserver with Master: 40fa985ef2bb423b82ae1f6caab8bb6e (127.30.146.130:41513)
I20260808 10:38:36.753754 17587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:43813
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:38:36.775960 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:40047
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:46101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:36.895450 17952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:36.895869 17952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:36.895925 17952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:36.900498 17952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:36.900579 17952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:36.900645 17952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:38:36.906405 17952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:46101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:40047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:36.907519 17952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:36.908582 17952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:36.918752 17957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:36.918957 17952 server_base.cc:1034] running on GCE node
W20260808 10:38:36.918829 17958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:36.918756 17960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:36.919507 17952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:36.920174 17952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:36.921370 17952 hybrid_clock.cc:648] HybridClock initialized: now 1786185516921357 us; error 23 us; skew 500 ppm
Aug 08 10:38:36 dist-test-slave-9gcw krb5kdc[17537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:36.925474 17952 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:38:36.926692 17952 webserver.cc:460] Webserver started at http://127.30.146.131:39969/ using document root <none> and password file <none>
I20260808 10:38:36.927434 17952 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:36.927542 17952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:36.927759 17952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:36.930428 17952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "c577fd915e1146eca493923dd261bf60"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "dbb4cea2593d78bfd26883e41f1f38cb"
server_key_iv: "e1c8785618004bbe0232bac31747a3ff"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.930998 17952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "c577fd915e1146eca493923dd261bf60"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "dbb4cea2593d78bfd26883e41f1f38cb"
server_key_iv: "e1c8785618004bbe0232bac31747a3ff"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.934832 17952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:38:36.937338 17967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:36.938657 17952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:38:36.938818 17952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "c577fd915e1146eca493923dd261bf60"
format_stamp: "Formatted at 2026-08-08 10:38:36 on dist-test-slave-9gcw"
server_key: "dbb4cea2593d78bfd26883e41f1f38cb"
server_key_iv: "e1c8785618004bbe0232bac31747a3ff"
server_key_version: "encryptionkey@0"
I20260808 10:38:36.938946 17952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:36.955696 17952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:36.958845 17952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:36.959064 17952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:36.959730 17952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:36.960608 17952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:36.960682 17952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:36.960788 17952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:36.960851 17952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:36.971226 17952 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:36299
I20260808 10:38:36.971262 18094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:36299 every 8 connection(s)
I20260808 10:38:36.972160 17952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:38:36.973802 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 17952
I20260808 10:38:36.973939 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:38:36.974290 31306 external_mini_cluster.cc:1428] Setting key f19ee48873175295f842a9ce353512e1
Aug 08 10:38:36 dist-test-slave-9gcw krb5kdc[17537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185516, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:36.987460 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:36.974293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:35891 (local address 127.30.146.190:40047)
0808 10:38:36.974490 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:36.974496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:36.975621 (+ 1125us) server_negotiation.cc:407] Connection header received
0808 10:38:36.977427 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:36.977430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:36.977487 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:36.977564 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:36.978817 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.979360 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.980537 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.980766 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.980892 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:36.983926 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:36.983929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:36.983931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:36.983956 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:36.985353 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:36.985868 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:36.985872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:36.985875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:36.985959 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:36.986310 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:36.986313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:36.986317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:36.986502 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:36.986614 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:36.987162 (+ 548us) server_negotiation.cc:299] Negotiation successful
0808 10:38:36.987290 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:38:36.988497 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:36.974778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:40047 (local address 127.30.146.131:35891)
0808 10:38:36.975430 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:36.975497 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:36.977212 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:36.977742 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:36.977749 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:36.978155 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:36.978661 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:36.978675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.979494 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.979498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:36.980404 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:36.980413 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:36.980931 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:36.980936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:36.981188 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:36.981946 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:36.981963 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:36.983743 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:36.985471 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:36.985477 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:36.985490 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:36.985755 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:36.986058 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:36.986061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:36.986063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:36.986209 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:36.986616 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:36.986623 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:36.986873 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 10:38:36.987104 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 10:38:36.987453 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":173,"threads_started":1}
I20260808 10:38:36.989774 18095 heartbeater.cc:347] Connected to a master server at 127.30.146.190:40047
I20260808 10:38:36.989998 18095 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:36.990454 18095 heartbeater.cc:511] Master 127.30.146.190:40047 requested a full tablet report, sending...
I20260808 10:38:36.991425 17587 ts_manager.cc:194] Registered new tserver with Master: c577fd915e1146eca493923dd261bf60 (127.30.146.131:36299)
I20260808 10:38:36.992005 17587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:35891
I20260808 10:38:37.001811 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:38:37.011616 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.004310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.30.146.190:40047)
0808 10:38:37.004512 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:37.004516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:37.004537 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:38:37.004677 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:37.004683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:37.004749 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:37.004834 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:37.005596 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.006128 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.006937 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.007146 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.007222 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.008541 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:37.008544 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:37.008546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:37.008570 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:37.010005 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:37.010407 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:37.010409 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:37.010410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:37.010484 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:37.010741 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:37.010744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:37.010745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:37.010976 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:37.011080 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:37.011285 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 10:38:37.011390 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 10:38:37.014639 17587 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:38:37.017778 17587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:38:37.033281 18109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.026727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.30.146.130:41513)
0808 10:38:37.027777 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:37.027790 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:37.028025 (+ 235us) server_negotiation.cc:407] Connection header received
0808 10:38:37.028116 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:37.028122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:37.028345 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:37.028513 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:37.028868 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.030184 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.031715 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.032607 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.032768 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.032915 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:37.032978 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 10:38:37.033082 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":160,"threads_started":1}
I20260808 10:38:37.033419 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.026952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:36299 (local address 127.0.0.1:46556)
0808 10:38:37.027919 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:37.027932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:37.028005 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:37.028736 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:37.028739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:37.028766 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:37.028983 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.028989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.030376 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.030379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.031733 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.031741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.032832 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.032836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.032947 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.032971 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:38:37.033023 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:38:37.033092 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":114,"thread_start_us":97,"threads_started":1}
I20260808 10:38:37.033715 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.027417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46556 (local address 127.30.146.131:36299)
0808 10:38:37.028252 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:37.028263 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:37.028284 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:38:37.028384 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:37.028390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:37.028573 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:37.028945 (+ 372us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:37.029089 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.030218 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.031862 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.032681 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.033258 (+ 577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.033406 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:37.033462 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 10:38:37.033574 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":165,"threads_started":1}
I20260808 10:38:37.033136 18106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.026556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41513 (local address 127.0.0.1:53886)
0808 10:38:37.027787 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:37.027803 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:37.027916 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:37.028521 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:37.028530 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:37.028560 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:37.028730 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.028741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.030359 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.030365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.031577 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.031587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.032783 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.032789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.032874 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.032894 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:38:37.032951 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 10:38:37.033003 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":84,"thread_start_us":108,"threads_started":1}
I20260808 10:38:37.033172 18107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.026633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36549 (local address 127.0.0.1:39410)
0808 10:38:37.027502 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:37.027624 (+ 122us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:37.027770 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:37.028755 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:37.028758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:37.028779 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:37.028951 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.028960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.030222 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.030227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.031594 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.031606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.032530 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.032533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.032642 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.032681 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:38:37.032807 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 10:38:37.032949 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":36,"thread_start_us":118,"threads_started":1}
I20260808 10:38:37.034315 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.027416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.30.146.129:36549)
0808 10:38:37.028130 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:37.028135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:37.028154 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:38:37.028294 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:37.028300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:37.028487 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:37.028625 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:37.029186 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.030048 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.031742 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.032395 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.032539 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.032694 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:37.033243 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 10:38:37.033347 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":96,"threads_started":1}
I20260808 10:38:37.036684 18024 tablet_service.cc:1467] Processing CreateTablet for tablet ae45ae4fa3324fb192ca6405155697ad (DEFAULT_TABLE table=test-table [id=04327ecb78c042498a111278a15e4a7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:37.036842 17870 tablet_service.cc:1467] Processing CreateTablet for tablet ae45ae4fa3324fb192ca6405155697ad (DEFAULT_TABLE table=test-table [id=04327ecb78c042498a111278a15e4a7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:37.036832 17730 tablet_service.cc:1467] Processing CreateTablet for tablet ae45ae4fa3324fb192ca6405155697ad (DEFAULT_TABLE table=test-table [id=04327ecb78c042498a111278a15e4a7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:37.037973 17870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae45ae4fa3324fb192ca6405155697ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:37.037973 17730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae45ae4fa3324fb192ca6405155697ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:37.038292 18024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae45ae4fa3324fb192ca6405155697ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:37.045125 18113 tablet_bootstrap.cc:492] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: Bootstrap starting.
I20260808 10:38:37.046522 18115 tablet_bootstrap.cc:492] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: Bootstrap starting.
I20260808 10:38:37.047571 18113 tablet_bootstrap.cc:654] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:37.048539 18115 tablet_bootstrap.cc:654] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:37.049055 18113 log.cc:824] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:37.049846 18114 tablet_bootstrap.cc:492] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240: Bootstrap starting.
I20260808 10:38:37.049961 18115 log.cc:824] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:37.051337 18113 tablet_bootstrap.cc:492] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: No bootstrap required, opened a new log
I20260808 10:38:37.051545 18113 ts_tablet_manager.cc:1397] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260808 10:38:37.052031 18114 tablet_bootstrap.cc:654] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:37.052246 18115 tablet_bootstrap.cc:492] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: No bootstrap required, opened a new log
I20260808 10:38:37.052443 18115 ts_tablet_manager.cc:1397] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:38:37.052870 18114 log.cc:824] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:37.054602 18114 tablet_bootstrap.cc:492] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240: No bootstrap required, opened a new log
I20260808 10:38:37.054811 18114 ts_tablet_manager.cc:1397] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:38:37.056084 18113 raft_consensus.cc:348] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 } } peers { permanent_uuid: "b34ba08087e0413dbc033ed665d2c240" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36549 } } peers { permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 } }
I20260808 10:38:37.056387 18113 raft_consensus.cc:374] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:37.056504 18113 raft_consensus.cc:729] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c577fd915e1146eca493923dd261bf60, State: Initialized, Role: FOLLOWER
I20260808 10:38:37.056699 18115 raft_consensus.cc:348] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 } } peers { permanent_uuid: "b34ba08087e0413dbc033ed665d2c240" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36549 } } peers { permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 } }
I20260808 10:38:37.056949 18115 raft_consensus.cc:374] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:37.057003 18115 raft_consensus.cc:729] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40fa985ef2bb423b82ae1f6caab8bb6e, State: Initialized, Role: FOLLOWER
I20260808 10:38:37.056996 18113 consensus_queue.cc:260] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 } } peers { permanent_uuid: "b34ba08087e0413dbc033ed665d2c240" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36549 } } peers { permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 } }
I20260808 10:38:37.057520 18115 consensus_queue.cc:260] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 } } peers { permanent_uuid: "b34ba08087e0413dbc033ed665d2c240" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36549 } } peers { permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 } }
I20260808 10:38:37.057773 18095 heartbeater.cc:503] Master 127.30.146.190:40047 was elected leader, sending a full tablet report...
I20260808 10:38:37.058104 18113 ts_tablet_manager.cc:1428] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260808 10:38:37.058436 18115 ts_tablet_manager.cc:1428] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:38:37.058758 17946 heartbeater.cc:503] Master 127.30.146.190:40047 was elected leader, sending a full tablet report...
I20260808 10:38:37.059458 18114 raft_consensus.cc:348] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 } } peers { permanent_uuid: "b34ba08087e0413dbc033ed665d2c240" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36549 } } peers { permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 } }
I20260808 10:38:37.059734 18114 raft_consensus.cc:374] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:37.059794 18114 raft_consensus.cc:729] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b34ba08087e0413dbc033ed665d2c240, State: Initialized, Role: FOLLOWER
I20260808 10:38:37.060242 18114 consensus_queue.cc:260] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 } } peers { permanent_uuid: "b34ba08087e0413dbc033ed665d2c240" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36549 } } peers { permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 } }
I20260808 10:38:37.060942 17798 heartbeater.cc:503] Master 127.30.146.190:40047 was elected leader, sending a full tablet report...
I20260808 10:38:37.061237 18114 ts_tablet_manager.cc:1428] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:38:37.089561 18122 raft_consensus.cc:482] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:37.089838 18122 raft_consensus.cc:504] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 } } peers { permanent_uuid: "b34ba08087e0413dbc033ed665d2c240" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36549 } } peers { permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 } }
I20260808 10:38:37.091152 18122 leader_election.cc:288] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40fa985ef2bb423b82ae1f6caab8bb6e (127.30.146.130:41513), c577fd915e1146eca493923dd261bf60 (127.30.146.131:36299)
I20260808 10:38:37.095436 18109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.091545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:36825 (local address 127.30.146.130:41513)
0808 10:38:37.091800 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:37.091805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:37.091961 (+ 156us) server_negotiation.cc:407] Connection header received
0808 10:38:37.092135 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:37.092137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:37.092182 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:37.092272 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:37.092575 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.093069 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.094321 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.094739 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.094869 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.094938 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:37.095191 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 10:38:37.095279 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260808 10:38:37.095436 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.091526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:60269 (local address 127.30.146.131:36299)
0808 10:38:37.091689 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:37.091694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:37.091921 (+ 227us) server_negotiation.cc:407] Connection header received
0808 10:38:37.092059 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:37.092062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:37.092121 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:37.092207 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:37.092545 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.093059 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.094329 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.094734 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.094869 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.094937 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:37.095188 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 10:38:37.095264 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:38:37.095496 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.091427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:36299 (local address 127.30.146.129:60269)
0808 10:38:37.091823 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:37.091839 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:37.091964 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:37.092219 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:37.092222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:37.092237 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:37.092422 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.092428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.093200 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.093206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.094183 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.094193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.094867 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.094871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.095057 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.095077 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:38:37.095168 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:38:37.095263 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":101,"spinlock_wait_cycles":32000,"thread_start_us":128,"threads_started":1}
I20260808 10:38:37.095500 18124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.091427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41513 (local address 127.30.146.129:36825)
0808 10:38:37.091849 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:37.091861 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:37.091998 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:37.092280 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:37.092283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:37.092323 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:37.092453 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.092459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.093222 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.093226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.094187 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.094197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.094867 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.094871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.095048 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.095071 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:38:37.095174 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:38:37.095280 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":124,"spinlock_wait_cycles":1408,"thread_start_us":122,"threads_started":1}
I20260808 10:38:37.096252 18046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae45ae4fa3324fb192ca6405155697ad" candidate_uuid: "b34ba08087e0413dbc033ed665d2c240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c577fd915e1146eca493923dd261bf60" is_pre_election: true
I20260808 10:38:37.096279 17895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae45ae4fa3324fb192ca6405155697ad" candidate_uuid: "b34ba08087e0413dbc033ed665d2c240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" is_pre_election: true
I20260808 10:38:37.096577 18046 raft_consensus.cc:2454] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b34ba08087e0413dbc033ed665d2c240 in term 0.
I20260808 10:38:37.096588 17895 raft_consensus.cc:2454] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b34ba08087e0413dbc033ed665d2c240 in term 0.
I20260808 10:38:37.097116 17674 leader_election.cc:302] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b34ba08087e0413dbc033ed665d2c240, c577fd915e1146eca493923dd261bf60; no voters:
I20260808 10:38:37.097505 18122 raft_consensus.cc:2781] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:38:37.097597 18122 raft_consensus.cc:482] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:38:37.097652 18122 raft_consensus.cc:3037] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:37.098811 18122 raft_consensus.cc:504] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 } } peers { permanent_uuid: "b34ba08087e0413dbc033ed665d2c240" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36549 } } peers { permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 } }
I20260808 10:38:37.099189 18122 leader_election.cc:288] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [CANDIDATE]: Term 1 election: Requested vote from peers 40fa985ef2bb423b82ae1f6caab8bb6e (127.30.146.130:41513), c577fd915e1146eca493923dd261bf60 (127.30.146.131:36299)
I20260808 10:38:37.099592 18046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae45ae4fa3324fb192ca6405155697ad" candidate_uuid: "b34ba08087e0413dbc033ed665d2c240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c577fd915e1146eca493923dd261bf60"
I20260808 10:38:37.099606 17895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae45ae4fa3324fb192ca6405155697ad" candidate_uuid: "b34ba08087e0413dbc033ed665d2c240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e"
I20260808 10:38:37.099736 17895 raft_consensus.cc:3037] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:37.099736 18046 raft_consensus.cc:3037] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:37.101135 17895 raft_consensus.cc:2454] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b34ba08087e0413dbc033ed665d2c240 in term 1.
I20260808 10:38:37.101166 18046 raft_consensus.cc:2454] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b34ba08087e0413dbc033ed665d2c240 in term 1.
I20260808 10:38:37.101473 17674 leader_election.cc:302] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b34ba08087e0413dbc033ed665d2c240, c577fd915e1146eca493923dd261bf60; no voters:
I20260808 10:38:37.101682 18122 raft_consensus.cc:2781] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:37.101950 18122 raft_consensus.cc:686] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [term 1 LEADER]: Becoming Leader. State: Replica: b34ba08087e0413dbc033ed665d2c240, State: Running, Role: LEADER
I20260808 10:38:37.102329 18122 consensus_queue.cc:237] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 } } peers { permanent_uuid: "b34ba08087e0413dbc033ed665d2c240" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36549 } } peers { permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 } }
I20260808 10:38:37.105895 17587 catalog_manager.cc:5495] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 reported cstate change: term changed from 0 to 1, leader changed from <none> to b34ba08087e0413dbc033ed665d2c240 (127.30.146.129). New cstate: current_term: 1 leader_uuid: "b34ba08087e0413dbc033ed665d2c240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b34ba08087e0413dbc033ed665d2c240" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 36549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 } health_report { overall_health: UNKNOWN } } }
I20260808 10:38:37.132072 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.128301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39418 (local address 127.30.146.129:36549)
0808 10:38:37.128478 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:37.128482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:37.128499 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:38:37.128593 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:37.128596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:37.128656 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:37.128772 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:38:37.129070 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.129684 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.130564 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.130775 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.130884 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.131207 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:38:37.131653 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:38:37.131793 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:37.131854 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 10:38:37.131915 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:38:37.138163 18046 raft_consensus.cc:1261] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60 [term 1 FOLLOWER]: Refusing update from remote peer b34ba08087e0413dbc033ed665d2c240: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:37.138163 17895 raft_consensus.cc:1261] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e [term 1 FOLLOWER]: Refusing update from remote peer b34ba08087e0413dbc033ed665d2c240: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:37.138891 18125 consensus_queue.cc:1035] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c577fd915e1146eca493923dd261bf60" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 36299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:37.139122 18122 consensus_queue.cc:1035] T ae45ae4fa3324fb192ca6405155697ad P b34ba08087e0413dbc033ed665d2c240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40fa985ef2bb423b82ae1f6caab8bb6e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:37.146831 18128 mvcc.cc:205] Tried to move back new op lower bound from 7316215878190346240 to 7316215878054522880. Current Snapshot: MvccSnapshot[applied={T|T < 7316215878190346240}]
I20260808 10:38:37.147703 18129 mvcc.cc:205] Tried to move back new op lower bound from 7316215878190346240 to 7316215878054522880. Current Snapshot: MvccSnapshot[applied={T|T < 7316215878190346240}]
I20260808 10:38:37.155982 17586 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:37.156178 17586 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:37.158744 17586 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e688b7f983584aa8a0cd229e91d0f392: Sending DeleteTablet for 3 replicas of tablet ae45ae4fa3324fb192ca6405155697ad
I20260808 10:38:37.159562 17870 tablet_service.cc:1514] Processing DeleteTablet for tablet ae45ae4fa3324fb192ca6405155697ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:37 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:53886
I20260808 10:38:37.159673 17730 tablet_service.cc:1514] Processing DeleteTablet for tablet ae45ae4fa3324fb192ca6405155697ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:37 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:39410
I20260808 10:38:37.159799 18024 tablet_service.cc:1514] Processing DeleteTablet for tablet ae45ae4fa3324fb192ca6405155697ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:37 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:46556
I20260808 10:38:37.160181 18138 tablet_replica.cc:330] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: stopping tablet replica
I20260808 10:38:37.160600 18138 raft_consensus.cc:2229] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:38:37.160826 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 17658
I20260808 10:38:37.164052 18138 raft_consensus.cc:2258] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:37.164542 18140 tablet_replica.cc:330] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: stopping tablet replica
I20260808 10:38:37.164896 18140 raft_consensus.cc:2229] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:38:37.165266 18140 raft_consensus.cc:2258] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:37.165946 18138 ts_tablet_manager.cc:1905] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:37.166666 18140 ts_tablet_manager.cc:1905] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:37.168965 18138 ts_tablet_manager.cc:1918] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:37.169080 18138 log.cc:1196] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/ae45ae4fa3324fb192ca6405155697ad
I20260808 10:38:37.169507 18138 ts_tablet_manager.cc:1939] T ae45ae4fa3324fb192ca6405155697ad P 40fa985ef2bb423b82ae1f6caab8bb6e: Deleting consensus metadata
I20260808 10:38:37.169560 18140 ts_tablet_manager.cc:1918] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:37.169654 18140 log.cc:1196] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/ae45ae4fa3324fb192ca6405155697ad
I20260808 10:38:37.169988 18140 ts_tablet_manager.cc:1939] T ae45ae4fa3324fb192ca6405155697ad P c577fd915e1146eca493923dd261bf60: Deleting consensus metadata
W20260808 10:38:37.170606 17571 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:36549 (error 108)
I20260808 10:38:37.170997 17573 catalog_manager.cc:4841] TS 40fa985ef2bb423b82ae1f6caab8bb6e (127.30.146.130:41513): tablet ae45ae4fa3324fb192ca6405155697ad (table test-table [id=04327ecb78c042498a111278a15e4a7a]) successfully deleted
I20260808 10:38:37.171147 17570 catalog_manager.cc:4841] TS c577fd915e1146eca493923dd261bf60 (127.30.146.131:36299): tablet ae45ae4fa3324fb192ca6405155697ad (table test-table [id=04327ecb78c042498a111278a15e4a7a]) successfully deleted
I20260808 10:38:37.171536 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 17803
I20260808 10:38:37.171900 18107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.171529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:36549 (local address 127.0.0.1:39428)
0808 10:38:37.171696 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:37.171783 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:36549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 10:38:37.174536 17571 catalog_manager.cc:4584] TS b34ba08087e0413dbc033ed665d2c240 (127.30.146.129:36549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae45ae4fa3324fb192ca6405155697ad: Network error: Client connection negotiation failed: client connection to 127.30.146.129:36549: connect: Connection refused (error 111)
I20260808 10:38:37.180191 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 17952
I20260808 10:38:37.188478 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 17553
2026-08-08T10:38:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2536 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T10:38:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:37Z Disabled control of system clock
I20260808 10:38:37.223815 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:37315
--webserver_interface=127.30.146.190
--webserver_port=0
--builtin_ntp_servers=127.30.146.148:43899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:37315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:38:37.345217 18148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:37.345476 18148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:37.345512 18148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:37.349625 18148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:37.349694 18148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:37.349709 18148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:37.349725 18148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:37.349741 18148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:37.354573 18148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:37315
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:37315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:37.355696 18148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:37.357051 18148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:37.366030 18157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:37.366088 18148 server_base.cc:1034] running on GCE node
W20260808 10:38:37.366039 18155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:37.366062 18154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:37.367003 18148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:37.368148 18148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:37.369318 18148 hybrid_clock.cc:648] HybridClock initialized: now 1786185517369293 us; error 40 us; skew 500 ppm
I20260808 10:38:37.371284 18148 webserver.cc:460] Webserver started at http://127.30.146.190:33767/ using document root <none> and password file <none>
I20260808 10:38:37.371840 18148 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:37.371902 18148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:37.372073 18148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:37.374156 18148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "5c6b6b73902e4c0e831827304e4606df"
format_stamp: "Formatted at 2026-08-08 10:38:37 on dist-test-slave-9gcw"
server_key: "8a73ae8493c75d636127d197f7e63cf3"
server_key_iv: "c6f5079e2e4525b381ab25813d4d0da6"
server_key_version: "encryptionkey@0"
I20260808 10:38:37.374671 18148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "5c6b6b73902e4c0e831827304e4606df"
format_stamp: "Formatted at 2026-08-08 10:38:37 on dist-test-slave-9gcw"
server_key: "8a73ae8493c75d636127d197f7e63cf3"
server_key_iv: "c6f5079e2e4525b381ab25813d4d0da6"
server_key_version: "encryptionkey@0"
I20260808 10:38:37.378400 18148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:38:37.381021 18164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:37.382397 18148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:38:37.382501 18148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "5c6b6b73902e4c0e831827304e4606df"
format_stamp: "Formatted at 2026-08-08 10:38:37 on dist-test-slave-9gcw"
server_key: "8a73ae8493c75d636127d197f7e63cf3"
server_key_iv: "c6f5079e2e4525b381ab25813d4d0da6"
server_key_version: "encryptionkey@0"
I20260808 10:38:37.382589 18148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:37.405332 18148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:37.406071 18148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:37.406232 18148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:37.413780 18148 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:37315
I20260808 10:38:37.413818 18218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:37315 every 8 connection(s)
I20260808 10:38:37.414834 18148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:37.418058 18220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:37.420557 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 18148
I20260808 10:38:37.420811 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:37.421223 31306 external_mini_cluster.cc:1428] Setting key a05984aeb9ed77494b0dfbbdddcc16d9
I20260808 10:38:37.423978 18220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df: Bootstrap starting.
I20260808 10:38:37.426421 18220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:37.427359 18220 log.cc:824] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:37.429915 18220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df: No bootstrap required, opened a new log
I20260808 10:38:37.431675 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.422916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.30.146.190:37315)
0808 10:38:37.423584 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:37.423604 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:37.423642 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:38:37.424429 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:37.424448 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:37.424895 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:37.425261 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:37.425633 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.426777 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.428009 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.428632 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.428770 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.429807 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:37.429812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:37.429824 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:37.430315 (+ 491us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:37.430372 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:37.430394 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:37.430477 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:37.430650 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 10:38:37.430948 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":155,"threads_started":1}
I20260808 10:38:37.434001 18220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6b6b73902e4c0e831827304e4606df" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37315 } }
I20260808 10:38:37.434322 18220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:37.434398 18220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c6b6b73902e4c0e831827304e4606df, State: Initialized, Role: FOLLOWER
I20260808 10:38:37.434808 18220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6b6b73902e4c0e831827304e4606df" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37315 } }
I20260808 10:38:37.434947 18220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:37.435014 18220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:37.435132 18220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:37.436440 18220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6b6b73902e4c0e831827304e4606df" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37315 } }
I20260808 10:38:37.436791 18220 leader_election.cc:302] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c6b6b73902e4c0e831827304e4606df; no voters:
I20260808 10:38:37.437114 18220 leader_election.cc:288] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:37.437288 18226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:37.437531 18226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [term 1 LEADER]: Becoming Leader. State: Replica: 5c6b6b73902e4c0e831827304e4606df, State: Running, Role: LEADER
I20260808 10:38:37.437840 18226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6b6b73902e4c0e831827304e4606df" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37315 } }
I20260808 10:38:37.438462 18220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:37.439596 18227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c6b6b73902e4c0e831827304e4606df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6b6b73902e4c0e831827304e4606df" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37315 } } }
I20260808 10:38:37.439584 18228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c6b6b73902e4c0e831827304e4606df. Latest consensus state: current_term: 1 leader_uuid: "5c6b6b73902e4c0e831827304e4606df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6b6b73902e4c0e831827304e4606df" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 37315 } } }
I20260808 10:38:37.439733 18227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:37.439733 18228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:37.440145 18236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:37.443395 18236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:37.449499 18236 catalog_manager.cc:1347] Generated new cluster ID: dcb0c9247c314d2dbdfb18660335421c
I20260808 10:38:37.449586 18236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:37.498908 18236 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:37.499898 18236 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:37.505303 18236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df: Generated new TSK 0
I20260808 10:38:37.506034 18236 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:37.512686 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37315
--builtin_ntp_servers=127.30.146.148:43899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:38:37.630398 18247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:37.630708 18247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:37.630751 18247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:37.634996 18247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:37.635056 18247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:37.635109 18247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:38:37.640051 18247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:37.641321 18247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:37.642719 18247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:37.653414 18255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:37.653398 18252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:37.653398 18253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:37.653513 18247 server_base.cc:1034] running on GCE node
I20260808 10:38:37.654178 18247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:37.654803 18247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:37.655966 18247 hybrid_clock.cc:648] HybridClock initialized: now 1786185517655937 us; error 44 us; skew 500 ppm
I20260808 10:38:37.657925 18247 webserver.cc:460] Webserver started at http://127.30.146.129:44833/ using document root <none> and password file <none>
I20260808 10:38:37.658499 18247 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:37.658561 18247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:37.658780 18247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:37.660851 18247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "dd6ac82a0c95462194b2f8e5d98316c2"
format_stamp: "Formatted at 2026-08-08 10:38:37 on dist-test-slave-9gcw"
server_key: "f03a87c3263db4c6d3c862ddc7bc7ae9"
server_key_iv: "03624ff8d00bae2000a94027b0228641"
server_key_version: "encryptionkey@0"
I20260808 10:38:37.661441 18247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "dd6ac82a0c95462194b2f8e5d98316c2"
format_stamp: "Formatted at 2026-08-08 10:38:37 on dist-test-slave-9gcw"
server_key: "f03a87c3263db4c6d3c862ddc7bc7ae9"
server_key_iv: "03624ff8d00bae2000a94027b0228641"
server_key_version: "encryptionkey@0"
I20260808 10:38:37.665252 18247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 10:38:37.667788 18261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:37.669070 18247 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:38:37.669296 18247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "dd6ac82a0c95462194b2f8e5d98316c2"
format_stamp: "Formatted at 2026-08-08 10:38:37 on dist-test-slave-9gcw"
server_key: "f03a87c3263db4c6d3c862ddc7bc7ae9"
server_key_iv: "03624ff8d00bae2000a94027b0228641"
server_key_version: "encryptionkey@0"
I20260808 10:38:37.669396 18247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:37.719817 18247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:37.720640 18247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:37.720844 18247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:37.721504 18247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:37.722481 18247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:37.722527 18247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:37.722591 18247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:37.722625 18247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:37.731715 18247 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:43553
I20260808 10:38:37.731755 18384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:43553 every 8 connection(s)
I20260808 10:38:37.732642 18247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:38:37.741269 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 18247
I20260808 10:38:37.741467 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:38:37.741910 31306 external_mini_cluster.cc:1428] Setting key da10ade90c179eecf9e248f7ed9650c3
I20260808 10:38:37.743234 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.734580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:53289 (local address 127.30.146.190:37315)
0808 10:38:37.734752 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:37.734757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:37.736011 (+ 1254us) server_negotiation.cc:407] Connection header received
0808 10:38:37.737462 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:37.737465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:37.737518 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:37.737605 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:37.738644 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.739340 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.740684 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.741085 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.741275 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.742570 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:37.742575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:37.742579 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:37.742602 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:37.742636 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:37.742653 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:37.742756 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:37.742971 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 10:38:37.743040 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:38:37.744079 18388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.735150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:37315 (local address 127.30.146.129:53289)
0808 10:38:37.735845 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:37.735886 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:37.737245 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:37.737796 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:37.737802 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:37.737886 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:37.738464 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.738486 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.739499 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.739501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.740486 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.740495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.741315 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.741319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.741509 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.742292 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:38:37.742294 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:37.742335 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:38:37.742399 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:37.742794 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:37.742804 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:37.742810 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:38:37.742944 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 10:38:37.743169 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":159,"threads_started":1}
I20260808 10:38:37.744517 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37315
--builtin_ntp_servers=127.30.146.148:43899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:38:37.745502 18385 heartbeater.cc:347] Connected to a master server at 127.30.146.190:37315
I20260808 10:38:37.745790 18385 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:37.746495 18385 heartbeater.cc:511] Master 127.30.146.190:37315 requested a full tablet report, sending...
I20260808 10:38:37.748250 18181 ts_manager.cc:194] Registered new tserver with Master: dd6ac82a0c95462194b2f8e5d98316c2 (127.30.146.129:43553)
I20260808 10:38:37.749652 18181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.129:53289
W20260808 10:38:37.866022 18389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:37.866334 18389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:37.866376 18389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:37.870549 18389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:37.870611 18389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:37.870697 18389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:38:37.875905 18389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:37.877224 18389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:37.878669 18389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:37.889042 18395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:37.889050 18398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:37.889039 18396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:37.889510 18389 server_base.cc:1034] running on GCE node
I20260808 10:38:37.889976 18389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:37.890673 18389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:37.891865 18389 hybrid_clock.cc:648] HybridClock initialized: now 1786185517891837 us; error 39 us; skew 500 ppm
I20260808 10:38:37.893985 18389 webserver.cc:460] Webserver started at http://127.30.146.130:42345/ using document root <none> and password file <none>
I20260808 10:38:37.894599 18389 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:37.894696 18389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:37.894912 18389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:37.897073 18389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "0456c62d34834a1582260d2c542ac72e"
format_stamp: "Formatted at 2026-08-08 10:38:37 on dist-test-slave-9gcw"
server_key: "f953528996d35be94b74df4eafffdbf3"
server_key_iv: "3ab31277592a20aee009c57b132ab19c"
server_key_version: "encryptionkey@0"
I20260808 10:38:37.897795 18389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "0456c62d34834a1582260d2c542ac72e"
format_stamp: "Formatted at 2026-08-08 10:38:37 on dist-test-slave-9gcw"
server_key: "f953528996d35be94b74df4eafffdbf3"
server_key_iv: "3ab31277592a20aee009c57b132ab19c"
server_key_version: "encryptionkey@0"
I20260808 10:38:37.901991 18389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:38:37.904767 18404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:37.906167 18389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:38:37.906340 18389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "0456c62d34834a1582260d2c542ac72e"
format_stamp: "Formatted at 2026-08-08 10:38:37 on dist-test-slave-9gcw"
server_key: "f953528996d35be94b74df4eafffdbf3"
server_key_iv: "3ab31277592a20aee009c57b132ab19c"
server_key_version: "encryptionkey@0"
I20260808 10:38:37.906466 18389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:37.929437 18389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:37.930172 18389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:37.930387 18389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:37.930999 18389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:37.932008 18389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:37.932087 18389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:37.932161 18389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:37.932221 18389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:37.941998 18389 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:42587
I20260808 10:38:37.942029 18531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:42587 every 8 connection(s)
I20260808 10:38:37.942950 18389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:38:37.944360 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 18389
I20260808 10:38:37.944458 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:38:37.944880 31306 external_mini_cluster.cc:1428] Setting key d37978a3bcf971c3615ef56485d5f1d9
I20260808 10:38:37.947144 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37315
--builtin_ntp_servers=127.30.146.148:43899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:38:37.954169 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.944896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:53707 (local address 127.30.146.190:37315)
0808 10:38:37.945332 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:37.945342 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:37.946889 (+ 1547us) server_negotiation.cc:407] Connection header received
0808 10:38:37.949064 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:37.949091 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:37.949194 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:37.949304 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:37.950301 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.950906 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.952089 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.952329 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.952445 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.953644 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:37.953647 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:37.953650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:37.953664 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:37.953698 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:37.953707 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:37.953791 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:37.953952 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 10:38:37.954016 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321}
I20260808 10:38:37.955090 18534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:37.945424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:37315 (local address 127.30.146.130:53707)
0808 10:38:37.946723 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:37.946781 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:37.948794 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:37.949504 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:37.949511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:37.949602 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:37.950140 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.950153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.951023 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.951025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.951947 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:37.951956 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:37.952495 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:37.952499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:37.952670 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:37.953371 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:38:37.953373 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:37.953414 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:38:37.953480 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:37.953793 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:37.953806 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:37.953814 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:38:37.953952 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 10:38:37.954260 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":140,"threads_started":1}
I20260808 10:38:37.956337 18532 heartbeater.cc:347] Connected to a master server at 127.30.146.190:37315
I20260808 10:38:37.956609 18532 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:37.957217 18532 heartbeater.cc:511] Master 127.30.146.190:37315 requested a full tablet report, sending...
I20260808 10:38:37.958408 18181 ts_manager.cc:194] Registered new tserver with Master: 0456c62d34834a1582260d2c542ac72e (127.30.146.130:42587)
I20260808 10:38:37.959347 18181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.130:53707
W20260808 10:38:38.066816 18536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:38.067112 18536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:38.067154 18536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:38.071385 18536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:38.071455 18536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:38.071516 18536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:38:38.076666 18536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:43899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:37315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:38.077836 18536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:38.079275 18536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:38.089438 18541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:38.089458 18542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:38.089534 18536 server_base.cc:1034] running on GCE node
W20260808 10:38:38.089449 18544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:38.090083 18536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:38.090744 18536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:38.091925 18536 hybrid_clock.cc:648] HybridClock initialized: now 1786185518091888 us; error 53 us; skew 500 ppm
I20260808 10:38:38.093935 18536 webserver.cc:460] Webserver started at http://127.30.146.131:35401/ using document root <none> and password file <none>
I20260808 10:38:38.094574 18536 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:38.094641 18536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:38.094875 18536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:38.097216 18536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "9a50e0dc96f64471bd673c8116691273"
format_stamp: "Formatted at 2026-08-08 10:38:38 on dist-test-slave-9gcw"
server_key: "cc20eeb190d72d3d756313dc8d40ab2a"
server_key_iv: "a051c752d8bcf15e59924f14df65e6e4"
server_key_version: "encryptionkey@0"
I20260808 10:38:38.097860 18536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "9a50e0dc96f64471bd673c8116691273"
format_stamp: "Formatted at 2026-08-08 10:38:38 on dist-test-slave-9gcw"
server_key: "cc20eeb190d72d3d756313dc8d40ab2a"
server_key_iv: "a051c752d8bcf15e59924f14df65e6e4"
server_key_version: "encryptionkey@0"
I20260808 10:38:38.101887 18536 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:38:38.104519 18551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:38.105854 18536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:38:38.105985 18536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "9a50e0dc96f64471bd673c8116691273"
format_stamp: "Formatted at 2026-08-08 10:38:38 on dist-test-slave-9gcw"
server_key: "cc20eeb190d72d3d756313dc8d40ab2a"
server_key_iv: "a051c752d8bcf15e59924f14df65e6e4"
server_key_version: "encryptionkey@0"
I20260808 10:38:38.106106 18536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:38.120288 18536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:38.121057 18536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:38.121351 18536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:38.121999 18536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:38.123008 18536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:38.123057 18536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:38.123121 18536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:38.123157 18536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:38.133613 18536 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:34789
I20260808 10:38:38.133634 18673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:34789 every 8 connection(s)
I20260808 10:38:38.134613 18536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:38:38.136852 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 18536
I20260808 10:38:38.136998 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:38:38.137359 31306 external_mini_cluster.cc:1428] Setting key e60ac49bbafd07175f4939f6a76a8100
I20260808 10:38:38.144865 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.136612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:46023 (local address 127.30.146.190:37315)
0808 10:38:38.136787 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:38.136794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:38.138107 (+ 1313us) server_negotiation.cc:407] Connection header received
0808 10:38:38.139905 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:38.139907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:38.139958 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:38.140024 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:38.140969 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.141574 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.142735 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.142962 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.143040 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.144397 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:38.144400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:38.144402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:38.144413 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:38.144430 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:38.144437 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:38.144493 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:38.144676 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 10:38:38.144728 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:38:38.145938 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.137107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:37315 (local address 127.30.146.131:46023)
0808 10:38:38.137934 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:38.137995 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:38.139666 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:38.140171 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:38.140178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:38.140335 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:38.140843 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.140856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.141693 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.141696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.142626 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.142636 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.143172 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.143176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.143349 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.144129 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:38:38.144131 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:38.144176 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:38:38.144249 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:38.144520 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:38.144528 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:38.144533 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:38:38.144668 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 10:38:38.144920 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":135,"threads_started":1}
I20260808 10:38:38.147153 18674 heartbeater.cc:347] Connected to a master server at 127.30.146.190:37315
I20260808 10:38:38.147434 18674 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:38.148051 18674 heartbeater.cc:511] Master 127.30.146.190:37315 requested a full tablet report, sending...
I20260808 10:38:38.149137 18181 ts_manager.cc:194] Registered new tserver with Master: 9a50e0dc96f64471bd673c8116691273 (127.30.146.131:34789)
I20260808 10:38:38.149804 18181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.131:46023
I20260808 10:38:38.151648 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:38:38.157756 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.153831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.30.146.190:37315)
0808 10:38:38.153999 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:38.154004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:38.154143 (+ 139us) server_negotiation.cc:407] Connection header received
0808 10:38:38.154298 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:38.154301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:38.154352 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:38.154433 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:38.154725 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.155243 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.156138 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.156366 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.156471 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.157325 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:38.157328 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:38.157333 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:38.157350 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:38.157368 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:38.157376 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:38.157458 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:38.157554 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 10:38:38.157609 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 10:38:38.160730 18181 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:38:38.163865 18181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:38:38.178853 18689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.172912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:34789 (local address 127.0.0.1:32906)
0808 10:38:38.173715 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:38.173728 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:38.173801 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:38.174443 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:38.174446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:38.174467 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:38.174608 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.174616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.176210 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.176213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.177349 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.177361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.178491 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.178494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.178581 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.178639 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 10:38:38.178686 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 10:38:38.178739 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":40,"thread_start_us":98,"threads_started":1}
I20260808 10:38:38.178896 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.172686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:42587 (local address 127.0.0.1:59200)
0808 10:38:38.173453 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:38.173465 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:38.173533 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:38.174499 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:38.174503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:38.174517 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:38.174630 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.174637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.176066 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.176070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.177456 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.177467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.178263 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.178266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.178398 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.178439 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 10:38:38.178568 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 10:38:38.178724 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":235,"spinlock_wait_cycles":768,"thread_start_us":113,"threads_started":1}
I20260808 10:38:38.178853 18687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.172493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:43553 (local address 127.0.0.1:46818)
0808 10:38:38.173313 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:38.173369 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:38.173502 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:38.174348 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:38.174354 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:38.174378 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:38.174723 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.174730 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.175734 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.175739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.177203 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.177216 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.178338 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.178342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.178448 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.178482 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 10:38:38.178530 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 10:38:38.178670 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":246,"thread_start_us":66,"threads_started":1}
I20260808 10:38:38.179450 18688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.172632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46818 (local address 127.30.146.129:43553)
0808 10:38:38.173804 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:38.173825 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:38.173853 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:38:38.173939 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:38.173944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:38.174165 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:38.174755 (+ 590us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:38.174906 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.175612 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.177324 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.178177 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.178944 (+ 767us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.179138 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:38.179187 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 10:38:38.179282 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":116,"threads_started":1}
I20260808 10:38:38.178902 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.173395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59200 (local address 127.30.146.130:42587)
0808 10:38:38.174072 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:38.174083 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:38.174105 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:38:38.174178 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:38.174184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:38.174384 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:38.174541 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:38.174781 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.175942 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.177583 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.178128 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.178277 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.178496 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:38.178600 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 10:38:38.178731 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":137,"threads_started":1}
I20260808 10:38:38.179661 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.173373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32906 (local address 127.30.146.131:34789)
0808 10:38:38.173943 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:38.173957 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:38.173982 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:38:38.174062 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:38.174067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:38.174273 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:38.174559 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:38.174710 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.175942 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.177477 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.178367 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.179183 (+ 816us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.179335 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:38.179383 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 10:38:38.179478 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":159,"threads_started":1}
I20260808 10:38:38.182370 18312 tablet_service.cc:1467] Processing CreateTablet for tablet 73d47100133a4dd28f840e7c51dc1cd6 (DEFAULT_TABLE table=test-table [id=9f58784018bb42ab81848ddf7f947427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:38.182394 18458 tablet_service.cc:1467] Processing CreateTablet for tablet 73d47100133a4dd28f840e7c51dc1cd6 (DEFAULT_TABLE table=test-table [id=9f58784018bb42ab81848ddf7f947427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:38.182399 18604 tablet_service.cc:1467] Processing CreateTablet for tablet 73d47100133a4dd28f840e7c51dc1cd6 (DEFAULT_TABLE table=test-table [id=9f58784018bb42ab81848ddf7f947427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:38.183403 18312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d47100133a4dd28f840e7c51dc1cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:38.183441 18604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d47100133a4dd28f840e7c51dc1cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:38.183441 18458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d47100133a4dd28f840e7c51dc1cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:38.191419 18695 tablet_bootstrap.cc:492] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: Bootstrap starting.
I20260808 10:38:38.191725 18693 tablet_bootstrap.cc:492] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2: Bootstrap starting.
I20260808 10:38:38.193480 18695 tablet_bootstrap.cc:654] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:38.193503 18694 tablet_bootstrap.cc:492] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: Bootstrap starting.
I20260808 10:38:38.193543 18693 tablet_bootstrap.cc:654] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:38.194334 18695 log.cc:824] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:38.194382 18693 log.cc:824] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:38.195515 18694 tablet_bootstrap.cc:654] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:38.196455 18694 log.cc:824] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:38.196612 18695 tablet_bootstrap.cc:492] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: No bootstrap required, opened a new log
I20260808 10:38:38.196657 18693 tablet_bootstrap.cc:492] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2: No bootstrap required, opened a new log
I20260808 10:38:38.196832 18693 ts_tablet_manager.cc:1397] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 10:38:38.196832 18695 ts_tablet_manager.cc:1397] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:38:38.198212 18694 tablet_bootstrap.cc:492] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: No bootstrap required, opened a new log
I20260808 10:38:38.198418 18694 ts_tablet_manager.cc:1397] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:38:38.201472 18695 raft_consensus.cc:348] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 } } peers { permanent_uuid: "0456c62d34834a1582260d2c542ac72e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42587 } } peers { permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 } }
I20260808 10:38:38.201472 18693 raft_consensus.cc:348] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 } } peers { permanent_uuid: "0456c62d34834a1582260d2c542ac72e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42587 } } peers { permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 } }
I20260808 10:38:38.201562 18694 raft_consensus.cc:348] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 } } peers { permanent_uuid: "0456c62d34834a1582260d2c542ac72e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42587 } } peers { permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 } }
I20260808 10:38:38.201766 18695 raft_consensus.cc:374] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:38.201766 18693 raft_consensus.cc:374] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:38.201785 18694 raft_consensus.cc:374] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:38.201835 18694 raft_consensus.cc:729] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a50e0dc96f64471bd673c8116691273, State: Initialized, Role: FOLLOWER
I20260808 10:38:38.201833 18693 raft_consensus.cc:729] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd6ac82a0c95462194b2f8e5d98316c2, State: Initialized, Role: FOLLOWER
I20260808 10:38:38.201865 18695 raft_consensus.cc:729] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0456c62d34834a1582260d2c542ac72e, State: Initialized, Role: FOLLOWER
I20260808 10:38:38.202325 18693 consensus_queue.cc:260] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 } } peers { permanent_uuid: "0456c62d34834a1582260d2c542ac72e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42587 } } peers { permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 } }
I20260808 10:38:38.202334 18695 consensus_queue.cc:260] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 } } peers { permanent_uuid: "0456c62d34834a1582260d2c542ac72e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42587 } } peers { permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 } }
I20260808 10:38:38.202342 18694 consensus_queue.cc:260] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 } } peers { permanent_uuid: "0456c62d34834a1582260d2c542ac72e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42587 } } peers { permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 } }
I20260808 10:38:38.203369 18695 ts_tablet_manager.cc:1428] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:38:38.203366 18693 ts_tablet_manager.cc:1428] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:38:38.203367 18694 ts_tablet_manager.cc:1428] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:38:38.203893 18385 heartbeater.cc:503] Master 127.30.146.190:37315 was elected leader, sending a full tablet report...
I20260808 10:38:38.203893 18532 heartbeater.cc:503] Master 127.30.146.190:37315 was elected leader, sending a full tablet report...
I20260808 10:38:38.204942 18674 heartbeater.cc:503] Master 127.30.146.190:37315 was elected leader, sending a full tablet report...
W20260808 10:38:38.233932 18386 tablet.cc:2369] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:38:38.284020 18699 raft_consensus.cc:482] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:38.284302 18699 raft_consensus.cc:504] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 } } peers { permanent_uuid: "0456c62d34834a1582260d2c542ac72e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42587 } } peers { permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 } }
I20260808 10:38:38.285815 18699 leader_election.cc:288] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd6ac82a0c95462194b2f8e5d98316c2 (127.30.146.129:43553), 9a50e0dc96f64471bd673c8116691273 (127.30.146.131:34789)
I20260808 10:38:38.289954 18688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.286123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:59945 (local address 127.30.146.129:43553)
0808 10:38:38.286285 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:38.286290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:38.286309 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:38:38.286368 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:38.286371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:38.286419 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:38.286500 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:38.286826 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.287558 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.288807 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.289402 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.289495 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.289535 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:38.289758 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 10:38:38.289817 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 10:38:38.289976 18534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.286009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:43553 (local address 127.30.146.130:59945)
0808 10:38:38.286142 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:38.286160 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:38.286282 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:38.286525 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:38.286529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:38.286555 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:38.286701 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.286713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.287669 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.287672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.288619 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.288630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.289516 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.289519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.289628 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.289655 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:38:38.289736 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 10:38:38.289819 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 10:38:38.290414 18702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.286107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:34789 (local address 127.30.146.130:39041)
0808 10:38:38.286620 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:38.286634 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:38.286784 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:38.287081 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:38.287083 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:38.287097 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:38.287192 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.287196 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.288119 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.288121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.289132 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:38.289169 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.289973 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.289979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:38.290084 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.290131 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 10:38:38.290223 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:38:38.290286 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":165,"threads_started":1}
I20260808 10:38:38.290446 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.286380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:39041 (local address 127.30.146.131:34789)
0808 10:38:38.286540 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:38.286545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:38.286716 (+ 171us) server_negotiation.cc:407] Connection header received
0808 10:38:38.286927 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:38.286931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:38.286980 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:38.287063 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:38.287317 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.287962 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.289280 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.289851 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.289961 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.290017 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:38.290239 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 10:38:38.290330 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:38:38.290607 18334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d47100133a4dd28f840e7c51dc1cd6" candidate_uuid: "0456c62d34834a1582260d2c542ac72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" is_pre_election: true
I20260808 10:38:38.291018 18334 raft_consensus.cc:2454] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0456c62d34834a1582260d2c542ac72e in term 0.
I20260808 10:38:38.291123 18625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d47100133a4dd28f840e7c51dc1cd6" candidate_uuid: "0456c62d34834a1582260d2c542ac72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a50e0dc96f64471bd673c8116691273" is_pre_election: true
I20260808 10:38:38.291446 18625 raft_consensus.cc:2454] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0456c62d34834a1582260d2c542ac72e in term 0.
I20260808 10:38:38.291539 18408 leader_election.cc:302] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0456c62d34834a1582260d2c542ac72e, dd6ac82a0c95462194b2f8e5d98316c2; no voters:
I20260808 10:38:38.291870 18699 raft_consensus.cc:2781] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:38:38.291987 18699 raft_consensus.cc:482] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:38:38.292049 18699 raft_consensus.cc:3037] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:38.293125 18699 raft_consensus.cc:504] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 } } peers { permanent_uuid: "0456c62d34834a1582260d2c542ac72e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42587 } } peers { permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 } }
I20260808 10:38:38.293612 18699 leader_election.cc:288] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [CANDIDATE]: Term 1 election: Requested vote from peers dd6ac82a0c95462194b2f8e5d98316c2 (127.30.146.129:43553), 9a50e0dc96f64471bd673c8116691273 (127.30.146.131:34789)
I20260808 10:38:38.294035 18334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d47100133a4dd28f840e7c51dc1cd6" candidate_uuid: "0456c62d34834a1582260d2c542ac72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6ac82a0c95462194b2f8e5d98316c2"
I20260808 10:38:38.294018 18625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d47100133a4dd28f840e7c51dc1cd6" candidate_uuid: "0456c62d34834a1582260d2c542ac72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a50e0dc96f64471bd673c8116691273"
I20260808 10:38:38.294162 18625 raft_consensus.cc:3037] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:38.294162 18334 raft_consensus.cc:3037] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:38.295548 18625 raft_consensus.cc:2454] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0456c62d34834a1582260d2c542ac72e in term 1.
I20260808 10:38:38.295548 18334 raft_consensus.cc:2454] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0456c62d34834a1582260d2c542ac72e in term 1.
I20260808 10:38:38.295941 18408 leader_election.cc:302] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0456c62d34834a1582260d2c542ac72e, 9a50e0dc96f64471bd673c8116691273, dd6ac82a0c95462194b2f8e5d98316c2; no voters:
I20260808 10:38:38.296190 18699 raft_consensus.cc:2781] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:38.296383 18699 raft_consensus.cc:686] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 1 LEADER]: Becoming Leader. State: Replica: 0456c62d34834a1582260d2c542ac72e, State: Running, Role: LEADER
I20260808 10:38:38.296761 18699 consensus_queue.cc:237] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 } } peers { permanent_uuid: "0456c62d34834a1582260d2c542ac72e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42587 } } peers { permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 } }
I20260808 10:38:38.300128 18179 catalog_manager.cc:5495] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0456c62d34834a1582260d2c542ac72e (127.30.146.130). New cstate: current_term: 1 leader_uuid: "0456c62d34834a1582260d2c542ac72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0456c62d34834a1582260d2c542ac72e" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 42587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 } health_report { overall_health: UNKNOWN } } }
I20260808 10:38:38.334960 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.331332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59208 (local address 127.30.146.130:42587)
0808 10:38:38.331489 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:38.331493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:38.331507 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:38:38.331663 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:38.331666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:38.331737 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:38.331840 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:38:38.332059 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.332671 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.333532 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:38.333766 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:38.333854 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:38.334125 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:38:38.334561 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:38:38.334655 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:38.334763 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 10:38:38.334817 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:38:38.341259 18625 raft_consensus.cc:1261] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273 [term 1 FOLLOWER]: Refusing update from remote peer 0456c62d34834a1582260d2c542ac72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:38.341341 18334 raft_consensus.cc:1261] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2 [term 1 FOLLOWER]: Refusing update from remote peer 0456c62d34834a1582260d2c542ac72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:38.342015 18699 consensus_queue.cc:1035] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a50e0dc96f64471bd673c8116691273" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:38.342243 18704 consensus_queue.cc:1035] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd6ac82a0c95462194b2f8e5d98316c2" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:38.350826 18709 mvcc.cc:205] Tried to move back new op lower bound from 7316215883117797376 to 7316215882946707456. Current Snapshot: MvccSnapshot[applied={T|T < 7316215883117797376}]
I20260808 10:38:38.351644 18708 mvcc.cc:205] Tried to move back new op lower bound from 7316215883117797376 to 7316215882946707456. Current Snapshot: MvccSnapshot[applied={T|T < 7316215883117797376}]
I20260808 10:38:38.358423 18180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:38.358644 18180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:38.361184 18180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c6b6b73902e4c0e831827304e4606df: Sending DeleteTablet for 3 replicas of tablet 73d47100133a4dd28f840e7c51dc1cd6
I20260808 10:38:38.362037 18458 tablet_service.cc:1514] Processing DeleteTablet for tablet 73d47100133a4dd28f840e7c51dc1cd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:38 UTC) from {username='slave'} at 127.0.0.1:59200
I20260808 10:38:38.362238 18312 tablet_service.cc:1514] Processing DeleteTablet for tablet 73d47100133a4dd28f840e7c51dc1cd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:38 UTC) from {username='slave'} at 127.0.0.1:46818
I20260808 10:38:38.362768 18718 tablet_replica.cc:330] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2: stopping tablet replica
I20260808 10:38:38.362779 18604 tablet_service.cc:1514] Processing DeleteTablet for tablet 73d47100133a4dd28f840e7c51dc1cd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:38 UTC) from {username='slave'} at 127.0.0.1:32906
I20260808 10:38:38.362874 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 18247
I20260808 10:38:38.363224 18718 raft_consensus.cc:2229] T 73d47100133a4dd28f840e7c51dc1cd6 P dd6ac82a0c95462194b2f8e5d98316c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:38:38.363409 18717 tablet_replica.cc:330] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: stopping tablet replica
I20260808 10:38:38.363767 18717 raft_consensus.cc:2229] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:38:38.364169 18717 raft_consensus.cc:2258] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:38.365890 18717 ts_tablet_manager.cc:1905] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:38.368891 18717 ts_tablet_manager.cc:1918] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:38.368999 18717 log.cc:1196] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/73d47100133a4dd28f840e7c51dc1cd6
I20260808 10:38:38.369388 18717 ts_tablet_manager.cc:1939] T 73d47100133a4dd28f840e7c51dc1cd6 P 0456c62d34834a1582260d2c542ac72e: Deleting consensus metadata
I20260808 10:38:38.370838 18165 catalog_manager.cc:4841] TS 0456c62d34834a1582260d2c542ac72e (127.30.146.130:42587): tablet 73d47100133a4dd28f840e7c51dc1cd6 (table test-table [id=9f58784018bb42ab81848ddf7f947427]) successfully deleted
I20260808 10:38:38.370613 18719 tablet_replica.cc:330] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: stopping tablet replica
I20260808 10:38:38.371461 18719 raft_consensus.cc:2229] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:38:38.371852 18719 raft_consensus.cc:2258] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 10:38:38.372278 18168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:43553 (error 108)
I20260808 10:38:38.372430 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 18389
I20260808 10:38:38.374487 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:38.374141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:43553 (local address 127.0.0.1:46834)
0808 10:38:38.374306 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:38.374385 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:43553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 10:38:38.374716 18168 catalog_manager.cc:4584] TS dd6ac82a0c95462194b2f8e5d98316c2 (127.30.146.129:43553): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73d47100133a4dd28f840e7c51dc1cd6: Network error: Client connection negotiation failed: client connection to 127.30.146.129:43553: connect: Connection refused (error 111)
I20260808 10:38:38.376368 18719 ts_tablet_manager.cc:1905] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:38.378890 18719 ts_tablet_manager.cc:1918] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:38.378978 18719 log.cc:1196] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/73d47100133a4dd28f840e7c51dc1cd6
I20260808 10:38:38.379311 18719 ts_tablet_manager.cc:1939] T 73d47100133a4dd28f840e7c51dc1cd6 P 9a50e0dc96f64471bd673c8116691273: Deleting consensus metadata
I20260808 10:38:38.380553 18166 catalog_manager.cc:4841] TS 9a50e0dc96f64471bd673c8116691273 (127.30.146.131:34789): tablet 73d47100133a4dd28f840e7c51dc1cd6 (table test-table [id=9f58784018bb42ab81848ddf7f947427]) successfully deleted
I20260808 10:38:38.382009 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 18536
I20260808 10:38:38.390980 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 18148
2026-08-08T10:38:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1202 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:38 dist-test-slave-9gcw krb5kdc[18725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:38 dist-test-slave-9gcw krb5kdc[18725](info): set up 2 sockets
Aug 08 10:38:38 dist-test-slave-9gcw krb5kdc[18725](info): commencing operation
krb5kdc: starting...
W20260808 10:38:39.517531 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:39 dist-test-slave-9gcw krb5kdc[18725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:38:39.656252 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:39043
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:39539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:39043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:39.772655 18739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:39.772994 18739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:39.773077 18739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:39.777197 18739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:39.777304 18739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:39.777341 18739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:39.777379 18739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:39.777428 18739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:39.783444 18739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:39539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:39043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:39043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:39.784601 18739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:39.785714 18739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:39.794919 18745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:39.794958 18747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:39.794958 18744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:39.795114 18739 server_base.cc:1034] running on GCE node
I20260808 10:38:39.795774 18739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:39.796996 18739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:39.798169 18739 hybrid_clock.cc:648] HybridClock initialized: now 1786185519798122 us; error 66 us; skew 500 ppm
Aug 08 10:38:39 dist-test-slave-9gcw krb5kdc[18725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185519, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:39.804803 18739 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:38:39.805891 18739 webserver.cc:460] Webserver started at http://127.30.146.190:44203/ using document root <none> and password file <none>
I20260808 10:38:39.806445 18739 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:39.806506 18739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:39.806682 18739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:39.808776 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "8825624a7ca246f7af620500be778800"
format_stamp: "Formatted at 2026-08-08 10:38:39 on dist-test-slave-9gcw"
server_key: "bab72eef85ddc88407ca42c17d6e5c80"
server_key_iv: "1694629a2efc4df118b0de4d42cd7a9b"
server_key_version: "encryptionkey@0"
I20260808 10:38:39.809399 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "8825624a7ca246f7af620500be778800"
format_stamp: "Formatted at 2026-08-08 10:38:39 on dist-test-slave-9gcw"
server_key: "bab72eef85ddc88407ca42c17d6e5c80"
server_key_iv: "1694629a2efc4df118b0de4d42cd7a9b"
server_key_version: "encryptionkey@0"
I20260808 10:38:39.813238 18739 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:38:39.815908 18756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:39.817498 18739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:38:39.817657 18739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "8825624a7ca246f7af620500be778800"
format_stamp: "Formatted at 2026-08-08 10:38:39 on dist-test-slave-9gcw"
server_key: "bab72eef85ddc88407ca42c17d6e5c80"
server_key_iv: "1694629a2efc4df118b0de4d42cd7a9b"
server_key_version: "encryptionkey@0"
I20260808 10:38:39.817787 18739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:39.832758 18739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:39.836390 18739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:39.836651 18739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:39.844864 18739 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:39043
I20260808 10:38:39.844894 18813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:39043 every 8 connection(s)
I20260808 10:38:39.845929 18739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:39.848773 18814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:39.854151 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 18739
I20260808 10:38:39.854275 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:39.854601 31306 external_mini_cluster.cc:1428] Setting key 909d04c5aff7e2ae2de068eb574476aa
I20260808 10:38:39.854841 18814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800: Bootstrap starting.
I20260808 10:38:39.857589 18814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:39.858369 18814 log.cc:824] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:39.860514 18814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800: No bootstrap required, opened a new log
Aug 08 10:38:39 dist-test-slave-9gcw krb5kdc[18725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:39.864881 18814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8825624a7ca246f7af620500be778800" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39043 } }
I20260808 10:38:39.865190 18814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:39.865270 18814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8825624a7ca246f7af620500be778800, State: Initialized, Role: FOLLOWER
I20260808 10:38:39.865866 18814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8825624a7ca246f7af620500be778800" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39043 } }
I20260808 10:38:39.866052 18814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:39.866154 18814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:39.866261 18814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:39.867265 18814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8825624a7ca246f7af620500be778800" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39043 } }
I20260808 10:38:39.867636 18814 leader_election.cc:302] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8825624a7ca246f7af620500be778800; no voters:
I20260808 10:38:39.868044 18814 leader_election.cc:288] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:39.868170 18821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:39.868432 18821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [term 1 LEADER]: Becoming Leader. State: Replica: 8825624a7ca246f7af620500be778800, State: Running, Role: LEADER
I20260808 10:38:39.868752 18821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8825624a7ca246f7af620500be778800" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39043 } }
I20260808 10:38:39.869495 18814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:39.870705 18824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8825624a7ca246f7af620500be778800. Latest consensus state: current_term: 1 leader_uuid: "8825624a7ca246f7af620500be778800" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8825624a7ca246f7af620500be778800" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39043 } } }
I20260808 10:38:39.870705 18823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8825624a7ca246f7af620500be778800" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8825624a7ca246f7af620500be778800" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 39043 } } }
I20260808 10:38:39.870869 18823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:39.870869 18824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:39.871382 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:39.856099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.30.146.190:39043)
0808 10:38:39.856593 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:39.856610 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:39.856647 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:38:39.857398 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:39.857416 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:39.857803 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:39.858166 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:39.859038 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:39.860093 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:39.861073 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:39.861649 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:39.861858 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:39.864767 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:39.864772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:39.864786 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:39.864818 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:39.866882 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:39.867385 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:39.867391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:39.867400 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:39.867551 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:39.867890 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:39.867894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:39.867896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:39.868508 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:39.869393 (+ 885us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:39.869556 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 10:38:39.869891 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":149,"threads_started":1}
W20260808 10:38:39.874913 18838 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:38:39.875013 18838 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:38:39.875088 18832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:39.875846 18832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:39.881798 18832 catalog_manager.cc:1347] Generated new cluster ID: be2f22832d1c4f628f9b01c0e56401be
I20260808 10:38:39.881882 18832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:39.892964 18832 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:39.894081 18832 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:39.903919 18832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800: Generated new TSK 0
I20260808 10:38:39.904716 18832 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:38:39.944509 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39043
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:39539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:40.072353 18845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:40.072660 18845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:40.072746 18845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:40.077234 18845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:40.077304 18845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:40.077400 18845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:38:40.083002 18845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:39539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:40.084219 18845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:40.085343 18845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:40.095558 18850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:40.095757 18845 server_base.cc:1034] running on GCE node
W20260808 10:38:40.095554 18851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:40.095522 18853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:40.096300 18845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:40.096913 18845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:40.098093 18845 hybrid_clock.cc:648] HybridClock initialized: now 1786185520098053 us; error 51 us; skew 500 ppm
Aug 08 10:38:40 dist-test-slave-9gcw krb5kdc[18725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185520, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:40.101140 18845 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:38:40.102180 18845 webserver.cc:460] Webserver started at http://127.30.146.129:39697/ using document root <none> and password file <none>
I20260808 10:38:40.102819 18845 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:40.102893 18845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:40.103117 18845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:40.105324 18845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "503b10195243422988d438b348977cf7"
format_stamp: "Formatted at 2026-08-08 10:38:40 on dist-test-slave-9gcw"
server_key: "68611cf36841e328e9cd4aac6bb8b4e0"
server_key_iv: "a0c6e292cdfa74b379b58fe9fedb59ad"
server_key_version: "encryptionkey@0"
I20260808 10:38:40.105929 18845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "503b10195243422988d438b348977cf7"
format_stamp: "Formatted at 2026-08-08 10:38:40 on dist-test-slave-9gcw"
server_key: "68611cf36841e328e9cd4aac6bb8b4e0"
server_key_iv: "a0c6e292cdfa74b379b58fe9fedb59ad"
server_key_version: "encryptionkey@0"
I20260808 10:38:40.109746 18845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:38:40.112260 18861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:40.113619 18845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:38:40.113718 18845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "503b10195243422988d438b348977cf7"
format_stamp: "Formatted at 2026-08-08 10:38:40 on dist-test-slave-9gcw"
server_key: "68611cf36841e328e9cd4aac6bb8b4e0"
server_key_iv: "a0c6e292cdfa74b379b58fe9fedb59ad"
server_key_version: "encryptionkey@0"
I20260808 10:38:40.113858 18845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:40.140513 18845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:40.143721 18845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:40.143955 18845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:40.144594 18845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:40.145524 18845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:40.145573 18845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:40.145643 18845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:40.145675 18845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:40.155061 18845 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:43891
I20260808 10:38:40.155089 18984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:43891 every 8 connection(s)
I20260808 10:38:40.156049 18845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:38:40.162693 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 18845
I20260808 10:38:40.162858 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:38:40.163254 31306 external_mini_cluster.cc:1428] Setting key 424b36d9426bc902c3e7608641929eca
Aug 08 10:38:40 dist-test-slave-9gcw krb5kdc[18725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185520, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:40.172173 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.158095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:39321 (local address 127.30.146.190:39043)
0808 10:38:40.158267 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:40.158272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:40.159208 (+ 936us) server_negotiation.cc:407] Connection header received
0808 10:38:40.160672 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:40.160675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:40.160724 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:40.160803 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:40.162202 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.162728 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.164389 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.164694 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.164793 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.167662 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:40.167666 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:40.167668 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:40.167692 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:40.169390 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:40.170200 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:40.170205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:40.170209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:40.170302 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:40.170830 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:40.170833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:40.170834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:40.171060 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:40.171169 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:40.171924 (+ 755us) server_negotiation.cc:299] Negotiation successful
0808 10:38:40.172044 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 10:38:40.173614 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.158558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39043 (local address 127.30.146.129:39321)
0808 10:38:40.159030 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:40.159079 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:40.160455 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:40.160989 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:40.160996 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:40.161492 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:40.162045 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.162061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.162914 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.162920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.164266 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.164278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.164875 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.164881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.165068 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.165827 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:40.165851 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:40.167467 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:40.169521 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:40.169531 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:40.169551 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:40.170091 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:40.170413 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:40.170419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:40.170425 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:40.170707 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:40.171189 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:40.171198 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:40.171617 (+ 419us) client_negotiation.cc:769] Sending connection context
0808 10:38:40.171897 (+ 280us) client_negotiation.cc:240] Negotiation successful
0808 10:38:40.172331 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":109,"threads_started":1}
I20260808 10:38:40.175050 18985 heartbeater.cc:347] Connected to a master server at 127.30.146.190:39043
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:38:40.175371 18985 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:40.176033 18985 heartbeater.cc:511] Master 127.30.146.190:39043 requested a full tablet report, sending...
I20260808 10:38:40.177722 18773 ts_manager.cc:194] Registered new tserver with Master: 503b10195243422988d438b348977cf7 (127.30.146.129:43891)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
I20260808 10:38:40.179018 18773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:39321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:38:40.197021 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39043
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:39539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:40.315552 18991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:40.315824 18991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:40.315868 18991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:40.320036 18991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:40.320098 18991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:40.320156 18991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:38:40.325652 18991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:39539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:40.326669 18991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:40.327644 18991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:40.337747 18999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:40.337765 18996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:40.337824 18997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:40.337916 18991 server_base.cc:1034] running on GCE node
I20260808 10:38:40.338496 18991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:40.339283 18991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:40.340468 18991 hybrid_clock.cc:648] HybridClock initialized: now 1786185520340438 us; error 48 us; skew 500 ppm
Aug 08 10:38:40 dist-test-slave-9gcw krb5kdc[18725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185520, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:40.343403 18991 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:38:40.344429 18991 webserver.cc:460] Webserver started at http://127.30.146.130:46379/ using document root <none> and password file <none>
I20260808 10:38:40.345016 18991 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:40.345111 18991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:40.345403 18991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:40.347458 18991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "412b0c7f6811413d9c0e05f733b98cc3"
format_stamp: "Formatted at 2026-08-08 10:38:40 on dist-test-slave-9gcw"
server_key: "d744444f8c07018f98feb1b661e79da4"
server_key_iv: "f979855862be29e01d88ec64f3021e6c"
server_key_version: "encryptionkey@0"
I20260808 10:38:40.348067 18991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "412b0c7f6811413d9c0e05f733b98cc3"
format_stamp: "Formatted at 2026-08-08 10:38:40 on dist-test-slave-9gcw"
server_key: "d744444f8c07018f98feb1b661e79da4"
server_key_iv: "f979855862be29e01d88ec64f3021e6c"
server_key_version: "encryptionkey@0"
I20260808 10:38:40.351926 18991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 10:38:40.354584 19006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:40.356014 18991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:38:40.356114 18991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "412b0c7f6811413d9c0e05f733b98cc3"
format_stamp: "Formatted at 2026-08-08 10:38:40 on dist-test-slave-9gcw"
server_key: "d744444f8c07018f98feb1b661e79da4"
server_key_iv: "f979855862be29e01d88ec64f3021e6c"
server_key_version: "encryptionkey@0"
I20260808 10:38:40.356210 18991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:40.382809 18991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:40.386406 18991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:40.386579 18991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:40.387169 18991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:40.388022 18991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:40.388080 18991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:40.388123 18991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:40.388144 18991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:40.397733 18991 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:46015
I20260808 10:38:40.397775 19129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:46015 every 8 connection(s)
I20260808 10:38:40.398669 18991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:38:40.405294 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 18991
I20260808 10:38:40.405437 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:38:40.405718 31306 external_mini_cluster.cc:1428] Setting key fd6e6e65a62d2ba5b2d49b9c4bcdb78e
Aug 08 10:38:40 dist-test-slave-9gcw krb5kdc[18725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185520, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:40.414710 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.400569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:33691 (local address 127.30.146.190:39043)
0808 10:38:40.400732 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:40.400736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:40.401938 (+ 1202us) server_negotiation.cc:407] Connection header received
0808 10:38:40.403727 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:40.403730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:40.403793 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:40.403876 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:40.405287 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.406063 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.407491 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.407726 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.407838 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.410484 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:40.410488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:40.410491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:40.410520 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:40.412196 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:40.412782 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:40.412789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:40.412794 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:40.412899 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:40.413340 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:40.413346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:40.413351 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:40.413620 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:40.413744 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:40.414269 (+ 525us) server_negotiation.cc:299] Negotiation successful
0808 10:38:40.414445 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:38:40.415429 19133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.401045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39043 (local address 127.30.146.130:33691)
0808 10:38:40.401710 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:40.401773 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:40.403492 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:40.404058 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:40.404067 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:40.404526 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:40.405075 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.405090 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.406204 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.406208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.407360 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.407370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.407893 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.407897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.408075 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.408711 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:40.408728 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:40.410299 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:40.412323 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:40.412328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:40.412340 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:40.412612 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:40.413005 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:40.413008 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:40.413009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:40.413222 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:40.413743 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:40.413748 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:40.413997 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 10:38:40.414153 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 10:38:40.414446 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":194,"threads_started":1}
I20260808 10:38:40.416734 19130 heartbeater.cc:347] Connected to a master server at 127.30.146.190:39043
I20260808 10:38:40.417023 19130 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:38:40.417708 19130 heartbeater.cc:511] Master 127.30.146.190:39043 requested a full tablet report, sending...
I20260808 10:38:40.418794 18773 ts_manager.cc:194] Registered new tserver with Master: 412b0c7f6811413d9c0e05f733b98cc3 (127.30.146.130:46015)
I20260808 10:38:40.419387 18773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:33691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:38:40.436280 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39043
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:39539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:40.554260 19136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:40.554559 19136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:40.554647 19136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:40.558873 19136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:40.558950 19136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:40.559051 19136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:38:40.564774 19136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:39539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:39043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:40.566011 19136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:40.567112 19136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:40.577585 19144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:40.577721 19142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:40.577724 19141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:40.577852 19136 server_base.cc:1034] running on GCE node
I20260808 10:38:40.578258 19136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:40.578899 19136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:40.580096 19136 hybrid_clock.cc:648] HybridClock initialized: now 1786185520580058 us; error 53 us; skew 500 ppm
Aug 08 10:38:40 dist-test-slave-9gcw krb5kdc[18725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185520, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:40.583328 19136 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:38:40.584282 19136 webserver.cc:460] Webserver started at http://127.30.146.131:38301/ using document root <none> and password file <none>
I20260808 10:38:40.584910 19136 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:40.585011 19136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:40.585309 19136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:40.587461 19136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "d0d2b140d32148fbb0710c4bd935de17"
format_stamp: "Formatted at 2026-08-08 10:38:40 on dist-test-slave-9gcw"
server_key: "279f79ef2a9f8fd8647070d0b6792934"
server_key_iv: "14ec09e83ebceff3caa5f75523b36422"
server_key_version: "encryptionkey@0"
I20260808 10:38:40.588044 19136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "d0d2b140d32148fbb0710c4bd935de17"
format_stamp: "Formatted at 2026-08-08 10:38:40 on dist-test-slave-9gcw"
server_key: "279f79ef2a9f8fd8647070d0b6792934"
server_key_iv: "14ec09e83ebceff3caa5f75523b36422"
server_key_version: "encryptionkey@0"
I20260808 10:38:40.592029 19136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:38:40.594668 19152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:40.596151 19136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:38:40.596345 19136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "d0d2b140d32148fbb0710c4bd935de17"
format_stamp: "Formatted at 2026-08-08 10:38:40 on dist-test-slave-9gcw"
server_key: "279f79ef2a9f8fd8647070d0b6792934"
server_key_iv: "14ec09e83ebceff3caa5f75523b36422"
server_key_version: "encryptionkey@0"
I20260808 10:38:40.596485 19136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:40.611613 19136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:40.615448 19136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:40.615621 19136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:40.616271 19136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:40.617233 19136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:40.617285 19136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:40.617328 19136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:40.617349 19136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:40.626978 19136 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:42605
I20260808 10:38:40.627033 19273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:42605 every 8 connection(s)
I20260808 10:38:40.627938 19136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:38:40.634112 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 19136
I20260808 10:38:40.634277 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:38:40.634584 31306 external_mini_cluster.cc:1428] Setting key 0db553c500b5a5f24e5a5afa9c53031e
Aug 08 10:38:40 dist-test-slave-9gcw krb5kdc[18725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185520, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:40.642824 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.630056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:45059 (local address 127.30.146.190:39043)
0808 10:38:40.630204 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:40.630208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:40.631024 (+ 816us) server_negotiation.cc:407] Connection header received
0808 10:38:40.632346 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:40.632350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:40.632400 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:40.632478 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:40.633751 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.634571 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.635836 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.636070 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.636184 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.639148 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:40.639151 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:40.639154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:40.639181 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:40.640531 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:40.641088 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:40.641092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:40.641094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:40.641236 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:40.641642 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:40.641647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:40.641648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:40.641858 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:40.641973 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:40.642508 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 10:38:40.642639 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:38:40.643721 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.630400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:39043 (local address 127.30.146.131:45059)
0808 10:38:40.630860 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:40.630903 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:40.632136 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:40.632647 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:40.632654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:40.633015 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:40.633607 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.633622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.634722 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.634730 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.635686 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.635697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.636241 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.636245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.636420 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.637088 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:40.637106 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:40.638943 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:40.640669 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:40.640676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:40.640693 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:40.640974 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:40.641339 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:40.641342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:40.641344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:40.641542 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:40.641974 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:40.641982 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:40.642257 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 10:38:40.642464 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 10:38:40.642810 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":119,"threads_started":1}
I20260808 10:38:40.644912 19274 heartbeater.cc:347] Connected to a master server at 127.30.146.190:39043
I20260808 10:38:40.645197 19274 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:40.645699 19274 heartbeater.cc:511] Master 127.30.146.190:39043 requested a full tablet report, sending...
I20260808 10:38:40.646708 18773 ts_manager.cc:194] Registered new tserver with Master: d0d2b140d32148fbb0710c4bd935de17 (127.30.146.131:42605)
I20260808 10:38:40.647351 18773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:45059
I20260808 10:38:40.648571 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:38:40.658486 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.650891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34394 (local address 127.30.146.190:39043)
0808 10:38:40.651050 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:40.651055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:40.651267 (+ 212us) server_negotiation.cc:407] Connection header received
0808 10:38:40.651365 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:40.651367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:40.651406 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:40.651473 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:40.652250 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.652751 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.653626 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.653853 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.653930 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.655540 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:40.655542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:40.655544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:40.655564 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:40.656823 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:40.657379 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:40.657382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:40.657383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:40.657458 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:40.657738 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:40.657740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:40.657741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:40.657935 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:40.658050 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:40.658258 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:38:40.658359 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:38:40.661484 18773 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:38:40.664505 18773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:38:40.679839 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.673280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.30.146.129:43891)
0808 10:38:40.673733 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:40.673750 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:40.673778 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:38:40.674568 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:40.674576 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:40.674798 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:40.674960 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:40.675332 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.676531 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.678238 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.679016 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.679135 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.679276 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:40.679561 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 10:38:40.679666 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":119,"threads_started":1}
I20260808 10:38:40.679966 19286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.673116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:43891 (local address 127.0.0.1:36596)
0808 10:38:40.673586 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:40.673640 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:40.673773 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:40.674963 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:40.674969 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:40.674993 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:40.675156 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.675167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.676654 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.676657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.678097 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.678108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.679135 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.679139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.679329 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.679392 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 10:38:40.679510 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 10:38:40.679775 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":234,"thread_start_us":110,"threads_started":1}
I20260808 10:38:40.679966 19289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.673765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:46015 (local address 127.0.0.1:57932)
0808 10:38:40.674491 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:40.674505 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:40.674598 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:40.675356 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:40.675362 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:40.675385 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:40.675466 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.675472 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.676801 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.676807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.677921 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.677930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.679135 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.679139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.679314 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.679414 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 10:38:40.679510 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 10:38:40.679773 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":180,"thread_start_us":109,"threads_started":1}
I20260808 10:38:40.680576 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.673407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57932 (local address 127.30.146.130:46015)
0808 10:38:40.674435 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:40.674448 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:40.674753 (+ 305us) server_negotiation.cc:407] Connection header received
0808 10:38:40.674847 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:40.674852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:40.675070 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:40.675230 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:40.675592 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.676660 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.678077 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.678991 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.679135 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.679276 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:40.680061 (+ 785us) server_negotiation.cc:299] Negotiation successful
0808 10:38:40.680436 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":917,"thread_start_us":133,"threads_started":1}
I20260808 10:38:40.681685 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.673924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:42605 (local address 127.0.0.1:52168)
0808 10:38:40.674365 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:40.674380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:40.674470 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:40.675153 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:40.675156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:40.675174 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:40.675282 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.675292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.676377 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.676382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.678098 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.678108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.681254 (+ 3146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.681258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.681384 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.681408 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:38:40.681486 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 10:38:40.681557 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":136,"thread_start_us":80,"threads_started":1}
I20260808 10:38:40.681789 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.674133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.30.146.131:42605)
0808 10:38:40.674719 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:40.674730 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:40.674752 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:38:40.674838 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:40.674844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:40.675029 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:40.675173 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:40.675449 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.676231 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.679340 (+ 3109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.680898 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.681028 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.681179 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:40.681537 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 10:38:40.681640 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":181,"threads_started":1}
I20260808 10:38:40.683305 19059 tablet_service.cc:1467] Processing CreateTablet for tablet fa0110cdab1b462d8afec15499550ee8 (DEFAULT_TABLE table=test-table [id=75e12cc0aca041e7a1fe9de61f88aa46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:40.683266 18913 tablet_service.cc:1467] Processing CreateTablet for tablet fa0110cdab1b462d8afec15499550ee8 (DEFAULT_TABLE table=test-table [id=75e12cc0aca041e7a1fe9de61f88aa46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:40.684415 19059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0110cdab1b462d8afec15499550ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:40.684422 18913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0110cdab1b462d8afec15499550ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:40.684357 19203 tablet_service.cc:1467] Processing CreateTablet for tablet fa0110cdab1b462d8afec15499550ee8 (DEFAULT_TABLE table=test-table [id=75e12cc0aca041e7a1fe9de61f88aa46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:40.685350 19203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0110cdab1b462d8afec15499550ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:40.691416 19292 tablet_bootstrap.cc:492] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7: Bootstrap starting.
I20260808 10:38:40.692819 19293 tablet_bootstrap.cc:492] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: Bootstrap starting.
I20260808 10:38:40.693485 19294 tablet_bootstrap.cc:492] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: Bootstrap starting.
I20260808 10:38:40.694121 19292 tablet_bootstrap.cc:654] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:40.694890 19292 log.cc:824] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:40.695078 19293 tablet_bootstrap.cc:654] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:40.695719 19294 tablet_bootstrap.cc:654] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:40.696504 19293 log.cc:824] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:40.696935 19294 log.cc:824] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:40.697686 19292 tablet_bootstrap.cc:492] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7: No bootstrap required, opened a new log
I20260808 10:38:40.697901 19292 ts_tablet_manager.cc:1397] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:38:40.698588 19293 tablet_bootstrap.cc:492] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: No bootstrap required, opened a new log
I20260808 10:38:40.698755 19294 tablet_bootstrap.cc:492] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: No bootstrap required, opened a new log
I20260808 10:38:40.698782 19293 ts_tablet_manager.cc:1397] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:38:40.698944 19294 ts_tablet_manager.cc:1397] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:38:40.702178 19294 raft_consensus.cc:348] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 } } peers { permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 } } peers { permanent_uuid: "d0d2b140d32148fbb0710c4bd935de17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42605 } }
I20260808 10:38:40.702430 19294 raft_consensus.cc:374] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:40.702484 19294 raft_consensus.cc:729] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0d2b140d32148fbb0710c4bd935de17, State: Initialized, Role: FOLLOWER
I20260808 10:38:40.702567 19292 raft_consensus.cc:348] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 } } peers { permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 } } peers { permanent_uuid: "d0d2b140d32148fbb0710c4bd935de17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42605 } }
I20260808 10:38:40.702863 19292 raft_consensus.cc:374] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:40.702917 19292 raft_consensus.cc:729] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 503b10195243422988d438b348977cf7, State: Initialized, Role: FOLLOWER
I20260808 10:38:40.702952 19294 consensus_queue.cc:260] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 } } peers { permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 } } peers { permanent_uuid: "d0d2b140d32148fbb0710c4bd935de17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42605 } }
I20260808 10:38:40.703008 19293 raft_consensus.cc:348] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 } } peers { permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 } } peers { permanent_uuid: "d0d2b140d32148fbb0710c4bd935de17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42605 } }
I20260808 10:38:40.703236 19293 raft_consensus.cc:374] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:40.703286 19293 raft_consensus.cc:729] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412b0c7f6811413d9c0e05f733b98cc3, State: Initialized, Role: FOLLOWER
I20260808 10:38:40.703380 19292 consensus_queue.cc:260] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 } } peers { permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 } } peers { permanent_uuid: "d0d2b140d32148fbb0710c4bd935de17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42605 } }
I20260808 10:38:40.703719 19293 consensus_queue.cc:260] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 } } peers { permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 } } peers { permanent_uuid: "d0d2b140d32148fbb0710c4bd935de17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42605 } }
I20260808 10:38:40.704113 19294 ts_tablet_manager.cc:1428] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:38:40.704335 19292 ts_tablet_manager.cc:1428] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:38:40.704514 19274 heartbeater.cc:503] Master 127.30.146.190:39043 was elected leader, sending a full tablet report...
I20260808 10:38:40.704581 19293 ts_tablet_manager.cc:1428] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:38:40.704681 18985 heartbeater.cc:503] Master 127.30.146.190:39043 was elected leader, sending a full tablet report...
I20260808 10:38:40.704909 19130 heartbeater.cc:503] Master 127.30.146.190:39043 was elected leader, sending a full tablet report...
I20260808 10:38:40.850374 19298 raft_consensus.cc:482] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:40.850612 19298 raft_consensus.cc:504] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 } } peers { permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 } } peers { permanent_uuid: "d0d2b140d32148fbb0710c4bd935de17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42605 } }
I20260808 10:38:40.851989 19298 leader_election.cc:288] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 503b10195243422988d438b348977cf7 (127.30.146.129:43891), 412b0c7f6811413d9c0e05f733b98cc3 (127.30.146.130:46015)
I20260808 10:38:40.857052 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.852339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:43891 (local address 127.30.146.131:33925)
0808 10:38:40.852572 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:40.852590 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:40.852729 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:40.853258 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:40.853262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:40.853288 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:40.853525 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.853535 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.854568 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.854572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.855756 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.855774 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.856617 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.856620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.856729 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.856756 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:38:40.856831 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:38:40.856906 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":3}
I20260808 10:38:40.857164 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.852616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:33925 (local address 127.30.146.129:43891)
0808 10:38:40.852766 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:40.852773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:40.852822 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:38:40.852901 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:40.852905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:40.852982 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:40.853165 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:40.853691 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.854455 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.855927 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.856431 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.856632 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.856702 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:40.856877 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:38:40.856971 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:38:40.857913 18934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0110cdab1b462d8afec15499550ee8" candidate_uuid: "d0d2b140d32148fbb0710c4bd935de17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "503b10195243422988d438b348977cf7" is_pre_election: true
I20260808 10:38:40.858165 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.852487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:41393 (local address 127.30.146.130:46015)
0808 10:38:40.852768 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:40.852775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:40.853380 (+ 605us) server_negotiation.cc:407] Connection header received
0808 10:38:40.853527 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:40.853531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:40.853603 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:40.853701 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:40.854113 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.854928 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.856522 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.857389 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.857524 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.857596 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:40.857936 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 10:38:40.858025 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:38:40.858182 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.852339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:46015 (local address 127.30.146.131:41393)
0808 10:38:40.853268 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:40.853281 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:40.853400 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:40.853717 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:40.853720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:40.853741 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:40.853908 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.853914 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.855073 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.855076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.856370 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:40.856380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.857683 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.857687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:40.857780 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.857803 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:38:40.857917 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 10:38:40.857998 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"mutex_wait_us":491,"spinlock_wait_cycles":40192,"thread_start_us":194,"threads_started":1}
I20260808 10:38:40.858244 18934 raft_consensus.cc:2454] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0d2b140d32148fbb0710c4bd935de17 in term 0.
I20260808 10:38:40.858873 19154 leader_election.cc:302] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 503b10195243422988d438b348977cf7, d0d2b140d32148fbb0710c4bd935de17; no voters:
I20260808 10:38:40.858863 19080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0110cdab1b462d8afec15499550ee8" candidate_uuid: "d0d2b140d32148fbb0710c4bd935de17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412b0c7f6811413d9c0e05f733b98cc3" is_pre_election: true
I20260808 10:38:40.859202 19298 raft_consensus.cc:2781] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:38:40.859208 19080 raft_consensus.cc:2454] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0d2b140d32148fbb0710c4bd935de17 in term 0.
I20260808 10:38:40.859314 19298 raft_consensus.cc:482] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:38:40.859378 19298 raft_consensus.cc:3037] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:40.860723 19298 raft_consensus.cc:504] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 } } peers { permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 } } peers { permanent_uuid: "d0d2b140d32148fbb0710c4bd935de17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42605 } }
I20260808 10:38:40.861207 19298 leader_election.cc:288] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [CANDIDATE]: Term 1 election: Requested vote from peers 503b10195243422988d438b348977cf7 (127.30.146.129:43891), 412b0c7f6811413d9c0e05f733b98cc3 (127.30.146.130:46015)
I20260808 10:38:40.861596 19080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0110cdab1b462d8afec15499550ee8" candidate_uuid: "d0d2b140d32148fbb0710c4bd935de17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412b0c7f6811413d9c0e05f733b98cc3"
I20260808 10:38:40.861652 18934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0110cdab1b462d8afec15499550ee8" candidate_uuid: "d0d2b140d32148fbb0710c4bd935de17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "503b10195243422988d438b348977cf7"
I20260808 10:38:40.861757 19080 raft_consensus.cc:3037] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:40.861794 18934 raft_consensus.cc:3037] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:40.863168 18934 raft_consensus.cc:2454] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0d2b140d32148fbb0710c4bd935de17 in term 1.
I20260808 10:38:40.863202 19080 raft_consensus.cc:2454] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0d2b140d32148fbb0710c4bd935de17 in term 1.
I20260808 10:38:40.863579 19153 leader_election.cc:302] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412b0c7f6811413d9c0e05f733b98cc3, d0d2b140d32148fbb0710c4bd935de17; no voters:
I20260808 10:38:40.863775 19298 raft_consensus.cc:2781] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:40.864044 19298 raft_consensus.cc:686] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 1 LEADER]: Becoming Leader. State: Replica: d0d2b140d32148fbb0710c4bd935de17, State: Running, Role: LEADER
I20260808 10:38:40.864395 19298 consensus_queue.cc:237] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 } } peers { permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 } } peers { permanent_uuid: "d0d2b140d32148fbb0710c4bd935de17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42605 } }
I20260808 10:38:40.867771 18772 catalog_manager.cc:5495] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0d2b140d32148fbb0710c4bd935de17 (127.30.146.131). New cstate: current_term: 1 leader_uuid: "d0d2b140d32148fbb0710c4bd935de17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0d2b140d32148fbb0710c4bd935de17" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 42605 } health_report { overall_health: HEALTHY } } }
W20260808 10:38:40.879019 19275 tablet.cc:2369] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:38:40.899947 19132 tablet.cc:2369] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:38:40.907557 18986 tablet.cc:2369] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:38:40.918443 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.914441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52182 (local address 127.30.146.131:42605)
0808 10:38:40.914607 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:40.914613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:40.914632 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:38:40.914718 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:40.914721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:40.914784 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:40.914884 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:38:40.915127 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.915700 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.916548 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:40.916842 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:40.916997 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:40.917368 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:38:40.917875 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:38:40.918007 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:40.918186 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 10:38:40.918238 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:38:40.925340 18934 raft_consensus.cc:1261] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7 [term 1 FOLLOWER]: Refusing update from remote peer d0d2b140d32148fbb0710c4bd935de17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:40.925375 19080 raft_consensus.cc:1261] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3 [term 1 FOLLOWER]: Refusing update from remote peer d0d2b140d32148fbb0710c4bd935de17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:40.926196 19298 consensus_queue.cc:1035] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "412b0c7f6811413d9c0e05f733b98cc3" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:40.926466 19302 consensus_queue.cc:1035] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "503b10195243422988d438b348977cf7" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:40.934039 19306 mvcc.cc:205] Tried to move back new op lower bound from 7316215893700423680 to 7316215893463171072. Current Snapshot: MvccSnapshot[applied={T|T < 7316215893700423680}]
I20260808 10:38:40.937114 19307 mvcc.cc:205] Tried to move back new op lower bound from 7316215893700423680 to 7316215893463171072. Current Snapshot: MvccSnapshot[applied={T|T < 7316215893700423680}]
I20260808 10:38:40.941529 19308 mvcc.cc:205] Tried to move back new op lower bound from 7316215893700423680 to 7316215893463171072. Current Snapshot: MvccSnapshot[applied={T|T < 7316215893700423680 or (T in {7316215893700423680})}]
I20260808 10:38:40.943957 18772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:40.944111 18772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:40.947126 18772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8825624a7ca246f7af620500be778800: Sending DeleteTablet for 3 replicas of tablet fa0110cdab1b462d8afec15499550ee8
I20260808 10:38:40.948060 19059 tablet_service.cc:1514] Processing DeleteTablet for tablet fa0110cdab1b462d8afec15499550ee8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:40 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:57932
I20260808 10:38:40.948145 18913 tablet_service.cc:1514] Processing DeleteTablet for tablet fa0110cdab1b462d8afec15499550ee8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:40 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:36596
I20260808 10:38:40.949096 19203 tablet_service.cc:1514] Processing DeleteTablet for tablet fa0110cdab1b462d8afec15499550ee8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:40 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:52168
I20260808 10:38:40.949438 19317 tablet_replica.cc:330] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7: stopping tablet replica
I20260808 10:38:40.949753 19318 tablet_replica.cc:330] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: stopping tablet replica
I20260808 10:38:40.949862 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 18845
I20260808 10:38:40.949936 19317 raft_consensus.cc:2229] T fa0110cdab1b462d8afec15499550ee8 P 503b10195243422988d438b348977cf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:38:40.950095 19318 raft_consensus.cc:2229] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:38:40.950649 19318 raft_consensus.cc:2258] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:40.952884 19318 ts_tablet_manager.cc:1905] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:40.954269 19316 tablet_replica.cc:330] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: stopping tablet replica
I20260808 10:38:40.954625 19316 raft_consensus.cc:2229] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:38:40.954977 19316 raft_consensus.cc:2258] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:40.956432 19318 ts_tablet_manager.cc:1918] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:40.956544 19318 log.cc:1196] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/fa0110cdab1b462d8afec15499550ee8
I20260808 10:38:40.956918 19318 ts_tablet_manager.cc:1939] T fa0110cdab1b462d8afec15499550ee8 P d0d2b140d32148fbb0710c4bd935de17: Deleting consensus metadata
I20260808 10:38:40.957325 19316 ts_tablet_manager.cc:1905] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:40.958204 18758 catalog_manager.cc:4841] TS d0d2b140d32148fbb0710c4bd935de17 (127.30.146.131:42605): tablet fa0110cdab1b462d8afec15499550ee8 (table test-table [id=75e12cc0aca041e7a1fe9de61f88aa46]) successfully deleted
I20260808 10:38:40.960093 19316 ts_tablet_manager.cc:1918] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:40.960217 19316 log.cc:1196] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/fa0110cdab1b462d8afec15499550ee8
I20260808 10:38:40.960527 19316 ts_tablet_manager.cc:1939] T fa0110cdab1b462d8afec15499550ee8 P 412b0c7f6811413d9c0e05f733b98cc3: Deleting consensus metadata
I20260808 10:38:40.961745 18757 catalog_manager.cc:4841] TS 412b0c7f6811413d9c0e05f733b98cc3 (127.30.146.130:46015): tablet fa0110cdab1b462d8afec15499550ee8 (table test-table [id=75e12cc0aca041e7a1fe9de61f88aa46]) successfully deleted
W20260808 10:38:40.963596 18758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:43891 (error 108)
I20260808 10:38:40.963828 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 18991
I20260808 10:38:40.964854 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:40.964497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:43891 (local address 127.0.0.1:36604)
0808 10:38:40.964680 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:40.964744 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:43891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260808 10:38:40.965049 18758 catalog_manager.cc:4584] TS 503b10195243422988d438b348977cf7 (127.30.146.129:43891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa0110cdab1b462d8afec15499550ee8: Network error: Client connection negotiation failed: client connection to 127.30.146.129:43891: connect: Connection refused (error 111)
I20260808 10:38:40.975085 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 19136
I20260808 10:38:40.983484 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 18739
2026-08-08T10:38:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2592 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T10:38:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:41Z Disabled control of system clock
I20260808 10:38:41.018553 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:32911
--webserver_interface=127.30.146.190
--webserver_port=0
--builtin_ntp_servers=127.30.146.148:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:32911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:38:41.138274 19324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:41.138599 19324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:41.138641 19324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:41.143126 19324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:41.143199 19324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:41.143215 19324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:41.143234 19324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:41.143249 19324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:41.148439 19324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:32911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:32911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:41.149587 19324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:41.150980 19324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:41.160029 19331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:41.160020 19332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:41.160017 19334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:41.160616 19324 server_base.cc:1034] running on GCE node
I20260808 10:38:41.161304 19324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:41.162534 19324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:41.163753 19324 hybrid_clock.cc:648] HybridClock initialized: now 1786185521163737 us; error 27 us; skew 500 ppm
I20260808 10:38:41.165959 19324 webserver.cc:460] Webserver started at http://127.30.146.190:43067/ using document root <none> and password file <none>
I20260808 10:38:41.166582 19324 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:41.166678 19324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:41.166923 19324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:41.169078 19324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "171333f98801477b99b66c2b18931d71"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "9b3e8eccac1387d8c4980a43562156d6"
server_key_iv: "ce6ac983d3c9810f0e7691cb2c8bd787"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.169704 19324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "171333f98801477b99b66c2b18931d71"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "9b3e8eccac1387d8c4980a43562156d6"
server_key_iv: "ce6ac983d3c9810f0e7691cb2c8bd787"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.173561 19324 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:38:41.176081 19340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:41.177455 19324 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:38:41.177603 19324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "171333f98801477b99b66c2b18931d71"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "9b3e8eccac1387d8c4980a43562156d6"
server_key_iv: "ce6ac983d3c9810f0e7691cb2c8bd787"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.177721 19324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:41.191993 19324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:41.192741 19324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:41.192966 19324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:41.200179 19324 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:32911
I20260808 10:38:41.200227 19397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:32911 every 8 connection(s)
I20260808 10:38:41.201217 19324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:41.204425 19398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:41.205499 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 19324
I20260808 10:38:41.205631 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:41.205996 31306 external_mini_cluster.cc:1428] Setting key b114a4e68639adf2eeb220697c0b7cfc
I20260808 10:38:41.210749 19398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71: Bootstrap starting.
I20260808 10:38:41.212941 19398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:41.213747 19398 log.cc:824] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:41.215575 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.207467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.30.146.190:32911)
0808 10:38:41.208167 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:41.208184 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:41.208221 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:38:41.208897 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:41.208913 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:41.209384 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:41.209721 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:41.210152 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.211171 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.212068 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.212537 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.212707 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.213662 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:41.213675 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:41.213687 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:41.214162 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:41.214234 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:41.214263 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:41.214385 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:41.214551 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 10:38:41.214859 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":139,"threads_started":1}
I20260808 10:38:41.215895 19398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71: No bootstrap required, opened a new log
I20260808 10:38:41.219403 19398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171333f98801477b99b66c2b18931d71" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32911 } }
I20260808 10:38:41.219653 19398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:41.219740 19398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 171333f98801477b99b66c2b18931d71, State: Initialized, Role: FOLLOWER
I20260808 10:38:41.220192 19398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171333f98801477b99b66c2b18931d71" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32911 } }
I20260808 10:38:41.220343 19398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:41.220431 19398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:41.220539 19398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:41.221536 19398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171333f98801477b99b66c2b18931d71" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32911 } }
I20260808 10:38:41.221871 19398 leader_election.cc:302] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 171333f98801477b99b66c2b18931d71; no voters:
I20260808 10:38:41.222182 19398 leader_election.cc:288] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:41.222308 19406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:41.222584 19406 raft_consensus.cc:686] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [term 1 LEADER]: Becoming Leader. State: Replica: 171333f98801477b99b66c2b18931d71, State: Running, Role: LEADER
I20260808 10:38:41.222887 19406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171333f98801477b99b66c2b18931d71" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32911 } }
I20260808 10:38:41.223368 19398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:41.224534 19408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 171333f98801477b99b66c2b18931d71. Latest consensus state: current_term: 1 leader_uuid: "171333f98801477b99b66c2b18931d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171333f98801477b99b66c2b18931d71" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32911 } } }
I20260808 10:38:41.224670 19408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:41.224979 19407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "171333f98801477b99b66c2b18931d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171333f98801477b99b66c2b18931d71" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 32911 } } }
I20260808 10:38:41.225093 19407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:41.225234 19415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:41.228472 19415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:41.233981 19415 catalog_manager.cc:1347] Generated new cluster ID: ba64b3f989224d018183b4a12d394e93
I20260808 10:38:41.234046 19415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:41.249598 19415 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:41.250552 19415 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:41.262841 19415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71: Generated new TSK 0
I20260808 10:38:41.263586 19415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:41.270859 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32911
--builtin_ntp_servers=127.30.146.148:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:38:41.398097 19425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:41.398362 19425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:41.398401 19425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:41.402362 19425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:41.402420 19425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:41.402474 19425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:38:41.408835 19425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:41.410007 19425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:41.411393 19425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:41.421322 19434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:41.421473 19425 server_base.cc:1034] running on GCE node
W20260808 10:38:41.421344 19430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:41.421376 19432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:41.422122 19425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:41.422801 19425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:41.423998 19425 hybrid_clock.cc:648] HybridClock initialized: now 1786185521423977 us; error 38 us; skew 500 ppm
I20260808 10:38:41.426090 19425 webserver.cc:460] Webserver started at http://127.30.146.129:35909/ using document root <none> and password file <none>
I20260808 10:38:41.426710 19425 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:41.426810 19425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:41.427029 19425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:41.429207 19425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "8ae88f227b4647fda2d587e7656c3d4f"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "7b3462705becfe639513aeb71249078d"
server_key_iv: "74db90e12f6dfda921c25d0a86889102"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.429821 19425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "8ae88f227b4647fda2d587e7656c3d4f"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "7b3462705becfe639513aeb71249078d"
server_key_iv: "74db90e12f6dfda921c25d0a86889102"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.434154 19425 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:38:41.436827 19440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:41.438213 19425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:38:41.438388 19425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "8ae88f227b4647fda2d587e7656c3d4f"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "7b3462705becfe639513aeb71249078d"
server_key_iv: "74db90e12f6dfda921c25d0a86889102"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.438550 19425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:41.463732 19425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:41.464509 19425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:41.464728 19425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:41.465430 19425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:41.466436 19425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:41.466513 19425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:41.466589 19425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:41.466634 19425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:41.476328 19425 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:43697
I20260808 10:38:41.476349 19562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:43697 every 8 connection(s)
I20260808 10:38:41.477329 19425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:38:41.479671 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 19425
I20260808 10:38:41.479817 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:38:41.480211 31306 external_mini_cluster.cc:1428] Setting key 511e485a71c6d449bf39849d38632da7
I20260808 10:38:41.482610 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32911
--builtin_ntp_servers=127.30.146.148:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:38:41.487875 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.479341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:59507 (local address 127.30.146.190:32911)
0808 10:38:41.479486 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:41.479490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:41.481021 (+ 1531us) server_negotiation.cc:407] Connection header received
0808 10:38:41.482767 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:41.482770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:41.482832 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:41.482922 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:41.483914 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.484495 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.485829 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.486062 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.486186 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.487382 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:41.487385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:41.487387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:41.487399 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:41.487416 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:41.487423 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:41.487502 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:41.487660 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:38:41.487727 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:38:41.488830 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.479894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:32911 (local address 127.30.146.129:59507)
0808 10:38:41.480853 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:41.480914 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:41.482544 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:41.483135 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:41.483144 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:41.483252 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:41.483754 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.483768 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.484637 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.484641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.485696 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.485707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.486233 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.486237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.486414 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.487106 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:38:41.487108 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:41.487148 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:38:41.487213 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:41.487505 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:41.487518 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:41.487524 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:38:41.487660 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 10:38:41.487962 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":155,"threads_started":1}
I20260808 10:38:41.490155 19563 heartbeater.cc:347] Connected to a master server at 127.30.146.190:32911
I20260808 10:38:41.490393 19563 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:41.490937 19563 heartbeater.cc:511] Master 127.30.146.190:32911 requested a full tablet report, sending...
I20260808 10:38:41.492424 19357 ts_manager.cc:194] Registered new tserver with Master: 8ae88f227b4647fda2d587e7656c3d4f (127.30.146.129:43697)
I20260808 10:38:41.493681 19357 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.129:59507
W20260808 10:38:41.609763 19566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:41.610134 19566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:41.610179 19566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:41.614418 19566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:41.614488 19566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:41.614545 19566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:38:41.619817 19566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:41.621038 19566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:41.622476 19566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:38:41.632794 19566 server_base.cc:1034] running on GCE node
W20260808 10:38:41.632745 19572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:41.632727 19571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:41.632728 19574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:41.633576 19566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:41.634461 19566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:41.635710 19566 hybrid_clock.cc:648] HybridClock initialized: now 1786185521635653 us; error 33 us; skew 500 ppm
I20260808 10:38:41.638072 19566 webserver.cc:460] Webserver started at http://127.30.146.130:37725/ using document root <none> and password file <none>
I20260808 10:38:41.638731 19566 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:41.638841 19566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:41.639110 19566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:41.641469 19566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "16526aaec736492793b467eb1afa13d6"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "23b9523bec1b2f0c712d6a0fcd5fcc37"
server_key_iv: "54ab974666d41300d7106c07063dc0c9"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.642127 19566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "16526aaec736492793b467eb1afa13d6"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "23b9523bec1b2f0c712d6a0fcd5fcc37"
server_key_iv: "54ab974666d41300d7106c07063dc0c9"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.646216 19566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:38:41.648852 19581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:41.650138 19566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:38:41.650290 19566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "16526aaec736492793b467eb1afa13d6"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "23b9523bec1b2f0c712d6a0fcd5fcc37"
server_key_iv: "54ab974666d41300d7106c07063dc0c9"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.650410 19566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:41.676795 19566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:41.677663 19566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:41.677955 19566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:41.678617 19566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:41.679792 19566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:41.679872 19566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:41.679952 19566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:41.679996 19566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:41.690056 19566 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:36031
I20260808 10:38:41.690081 19707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:36031 every 8 connection(s)
I20260808 10:38:41.691059 19566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:38:41.691636 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 19566
I20260808 10:38:41.691778 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:38:41.692159 31306 external_mini_cluster.cc:1428] Setting key 09937811c63105265b074025e775e61d
I20260808 10:38:41.694464 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32911
--builtin_ntp_servers=127.30.146.148:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:38:41.703387 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.693031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:40683 (local address 127.30.146.190:32911)
0808 10:38:41.693230 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:41.693234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:41.694610 (+ 1376us) server_negotiation.cc:407] Connection header received
0808 10:38:41.696848 (+ 2238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:41.696852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:41.696909 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:41.697008 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:41.698237 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.698785 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.700778 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.701031 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.701244 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.702736 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:41.702739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:41.702741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:41.702754 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:41.702774 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:41.702782 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:41.702845 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:41.703122 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 10:38:41.703183 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 10:38:41.704762 19711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.693568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:32911 (local address 127.30.146.130:40683)
0808 10:38:41.694431 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:41.694492 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:41.696588 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:41.697321 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:41.697328 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:41.697417 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:41.698061 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.698083 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.698937 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.698943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.700610 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.700625 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.701278 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.701285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.701562 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.702371 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:38:41.702374 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:41.702437 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:38:41.702538 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:41.702882 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:41.702897 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:41.702906 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:38:41.703054 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 10:38:41.703423 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":156,"threads_started":1}
I20260808 10:38:41.706245 19708 heartbeater.cc:347] Connected to a master server at 127.30.146.190:32911
I20260808 10:38:41.706532 19708 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:41.707199 19708 heartbeater.cc:511] Master 127.30.146.190:32911 requested a full tablet report, sending...
I20260808 10:38:41.708422 19357 ts_manager.cc:194] Registered new tserver with Master: 16526aaec736492793b467eb1afa13d6 (127.30.146.130:36031)
I20260808 10:38:41.709247 19357 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.130:40683
W20260808 10:38:41.814532 19712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:41.814862 19712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:41.814942 19712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:41.819008 19712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:41.819098 19712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:41.819195 19712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:38:41.824254 19712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:32911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:41.825505 19712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:41.826964 19712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:41.837213 19720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:41.837225 19718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:41.837376 19712 server_base.cc:1034] running on GCE node
W20260808 10:38:41.837204 19717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:41.837877 19712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:41.838523 19712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:41.839767 19712 hybrid_clock.cc:648] HybridClock initialized: now 1786185521839725 us; error 47 us; skew 500 ppm
I20260808 10:38:41.841717 19712 webserver.cc:460] Webserver started at http://127.30.146.131:46065/ using document root <none> and password file <none>
I20260808 10:38:41.842427 19712 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:41.842499 19712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:41.842736 19712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:41.844965 19712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "8b5caee635da4ebaac7c7cf8d47acfa1"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "770eb9ad57444253f8ac2e2c8bdb74cc"
server_key_iv: "05f87809fe659df158ea4841bf346c7d"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.845587 19712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "8b5caee635da4ebaac7c7cf8d47acfa1"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "770eb9ad57444253f8ac2e2c8bdb74cc"
server_key_iv: "05f87809fe659df158ea4841bf346c7d"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.849341 19712 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:38:41.851850 19726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:41.853211 19712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:38:41.853322 19712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "8b5caee635da4ebaac7c7cf8d47acfa1"
format_stamp: "Formatted at 2026-08-08 10:38:41 on dist-test-slave-9gcw"
server_key: "770eb9ad57444253f8ac2e2c8bdb74cc"
server_key_iv: "05f87809fe659df158ea4841bf346c7d"
server_key_version: "encryptionkey@0"
I20260808 10:38:41.853453 19712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:41.868904 19712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:41.869765 19712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:41.869988 19712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:41.870645 19712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:41.871649 19712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:41.871706 19712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:41.871781 19712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:41.871826 19712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:41.880901 19712 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:34461
I20260808 10:38:41.881064 19850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:34461 every 8 connection(s)
I20260808 10:38:41.881945 19712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:38:41.882850 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 19712
I20260808 10:38:41.882972 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:38:41.883285 31306 external_mini_cluster.cc:1428] Setting key 5d2493877d6e6879d2860406a1f15ee6
I20260808 10:38:41.891919 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.884098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:36947 (local address 127.30.146.190:32911)
0808 10:38:41.884248 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:41.884252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:41.885586 (+ 1334us) server_negotiation.cc:407] Connection header received
0808 10:38:41.887318 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:41.887321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:41.887367 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:41.887436 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:41.888310 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.888823 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.890039 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.890261 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.890340 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.891477 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:41.891480 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:41.891482 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:41.891495 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:41.891515 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:41.891523 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:41.891582 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:41.891740 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:38:41.891798 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 10:38:41.892851 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.884550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:32911 (local address 127.30.146.131:36947)
0808 10:38:41.885418 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:41.885479 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:41.887117 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:41.887596 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:41.887602 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:41.887687 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:41.888184 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.888196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.888934 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.888936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.889930 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.889940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.890408 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.890411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.890558 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.891180 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:38:41.891182 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:41.891242 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:38:41.891319 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:41.891608 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:41.891615 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:41.891619 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:38:41.891717 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:38:41.891951 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":130,"threads_started":1}
I20260808 10:38:41.894150 19851 heartbeater.cc:347] Connected to a master server at 127.30.146.190:32911
I20260808 10:38:41.894409 19851 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:41.894953 19851 heartbeater.cc:511] Master 127.30.146.190:32911 requested a full tablet report, sending...
I20260808 10:38:41.896034 19357 ts_manager.cc:194] Registered new tserver with Master: 8b5caee635da4ebaac7c7cf8d47acfa1 (127.30.146.131:34461)
I20260808 10:38:41.896631 19357 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.131:36947
I20260808 10:38:41.898563 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:38:41.904615 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.900652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60784 (local address 127.30.146.190:32911)
0808 10:38:41.900810 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:41.900817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:41.900970 (+ 153us) server_negotiation.cc:407] Connection header received
0808 10:38:41.901065 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:41.901068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:41.901108 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:41.901274 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:41.901556 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.902051 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.902886 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.903120 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.903221 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.904055 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:41.904058 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:41.904060 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:41.904071 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:41.904090 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:41.904098 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:41.904206 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:41.904302 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 10:38:41.904336 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:38:41.907894 19357 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:38:41.911131 19357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:38:41.925458 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.919677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:43697 (local address 127.0.0.1:34692)
0808 10:38:41.920044 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:41.920096 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:41.920231 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:41.920971 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:41.920977 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:41.921002 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:41.921185 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.921193 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.922383 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.922389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.923805 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.923813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.924876 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.924880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.924992 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.925031 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:38:41.925166 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 10:38:41.925290 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":35,"thread_start_us":72,"threads_started":1}
I20260808 10:38:41.925835 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.919842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.30.146.129:43697)
0808 10:38:41.920228 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:41.920241 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:41.920271 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:38:41.920361 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:41.920367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:41.920587 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:41.920749 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:41.921336 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.922274 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.923935 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.924743 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.924871 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.925064 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:41.925206 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 10:38:41.925334 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":142,"threads_started":1}
I20260808 10:38:41.926111 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.920414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:36031 (local address 127.0.0.1:50840)
0808 10:38:41.920758 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:41.920774 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:41.920859 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:41.922037 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:41.922040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:41.922059 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:41.922141 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.922147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.923373 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.923376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.924502 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.924511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.925691 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.925695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.925795 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.925822 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:38:41.925873 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:38:41.925967 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260808 10:38:41.926404 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.920731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:34461 (local address 127.0.0.1:52046)
0808 10:38:41.921229 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:41.921242 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:41.921327 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:41.922223 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:41.922228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:41.922247 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:41.922343 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.922349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.923802 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.923805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.924931 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:41.924942 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.926051 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.926054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:41.926143 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.926167 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:38:41.926218 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:38:41.926282 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":93,"threads_started":1}
I20260808 10:38:41.926864 19867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.920618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.30.146.130:36031)
0808 10:38:41.921570 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:41.921585 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:41.921609 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:38:41.921692 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:41.921697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:41.921877 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:41.922039 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:41.922264 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.923257 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.924626 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.925545 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.926350 (+ 805us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.926543 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:41.926592 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 10:38:41.926706 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":105,"threads_started":1}
I20260808 10:38:41.927217 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:41.921212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52046 (local address 127.30.146.131:34461)
0808 10:38:41.921643 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:41.921654 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:41.921675 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:38:41.921752 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:41.921757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:41.921897 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:41.922039 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:41.922456 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.923657 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.925072 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:41.925897 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:41.926692 (+ 795us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:41.926871 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:41.926937 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 10:38:41.927031 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":154,"threads_started":1}
I20260808 10:38:41.929050 19493 tablet_service.cc:1467] Processing CreateTablet for tablet f5c3333c01f843c597d7fb3b92fd1afd (DEFAULT_TABLE table=test-table [id=924f57281d39431ba8588756331ee561]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:41.929571 19635 tablet_service.cc:1467] Processing CreateTablet for tablet f5c3333c01f843c597d7fb3b92fd1afd (DEFAULT_TABLE table=test-table [id=924f57281d39431ba8588756331ee561]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:41.930261 19493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5c3333c01f843c597d7fb3b92fd1afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:41.929872 19781 tablet_service.cc:1467] Processing CreateTablet for tablet f5c3333c01f843c597d7fb3b92fd1afd (DEFAULT_TABLE table=test-table [id=924f57281d39431ba8588756331ee561]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:41.930579 19635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5c3333c01f843c597d7fb3b92fd1afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:41.930838 19781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5c3333c01f843c597d7fb3b92fd1afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:41.936890 19870 tablet_bootstrap.cc:492] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: Bootstrap starting.
I20260808 10:38:41.937565 19871 tablet_bootstrap.cc:492] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f: Bootstrap starting.
I20260808 10:38:41.938942 19872 tablet_bootstrap.cc:492] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: Bootstrap starting.
I20260808 10:38:41.939550 19870 tablet_bootstrap.cc:654] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:41.939550 19871 tablet_bootstrap.cc:654] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:41.940505 19871 log.cc:824] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:41.940505 19870 log.cc:824] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:41.940709 19872 tablet_bootstrap.cc:654] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:41.941597 19872 log.cc:824] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:41.943117 19871 tablet_bootstrap.cc:492] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f: No bootstrap required, opened a new log
I20260808 10:38:41.943114 19870 tablet_bootstrap.cc:492] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: No bootstrap required, opened a new log
I20260808 10:38:41.943328 19870 ts_tablet_manager.cc:1397] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 10:38:41.943328 19871 ts_tablet_manager.cc:1397] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:38:41.947882 19870 raft_consensus.cc:348] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae88f227b4647fda2d587e7656c3d4f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43697 } } peers { permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 } } peers { permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 } }
I20260808 10:38:41.947882 19871 raft_consensus.cc:348] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae88f227b4647fda2d587e7656c3d4f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43697 } } peers { permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 } } peers { permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 } }
I20260808 10:38:41.948176 19871 raft_consensus.cc:374] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:41.948167 19870 raft_consensus.cc:374] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:41.948226 19870 raft_consensus.cc:729] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16526aaec736492793b467eb1afa13d6, State: Initialized, Role: FOLLOWER
I20260808 10:38:41.948236 19871 raft_consensus.cc:729] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae88f227b4647fda2d587e7656c3d4f, State: Initialized, Role: FOLLOWER
I20260808 10:38:41.948722 19870 consensus_queue.cc:260] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae88f227b4647fda2d587e7656c3d4f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43697 } } peers { permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 } } peers { permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 } }
I20260808 10:38:41.948722 19871 consensus_queue.cc:260] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae88f227b4647fda2d587e7656c3d4f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43697 } } peers { permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 } } peers { permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 } }
I20260808 10:38:41.949818 19871 ts_tablet_manager.cc:1428] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:38:41.949818 19870 ts_tablet_manager.cc:1428] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 10:38:41.950172 19563 heartbeater.cc:503] Master 127.30.146.190:32911 was elected leader, sending a full tablet report...
I20260808 10:38:41.950171 19708 heartbeater.cc:503] Master 127.30.146.190:32911 was elected leader, sending a full tablet report...
I20260808 10:38:41.951782 19872 tablet_bootstrap.cc:492] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: No bootstrap required, opened a new log
I20260808 10:38:41.952011 19872 ts_tablet_manager.cc:1397] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260808 10:38:41.956363 19872 raft_consensus.cc:348] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae88f227b4647fda2d587e7656c3d4f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43697 } } peers { permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 } } peers { permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 } }
I20260808 10:38:41.956599 19872 raft_consensus.cc:374] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:41.956642 19872 raft_consensus.cc:729] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5caee635da4ebaac7c7cf8d47acfa1, State: Initialized, Role: FOLLOWER
I20260808 10:38:41.957034 19872 consensus_queue.cc:260] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae88f227b4647fda2d587e7656c3d4f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43697 } } peers { permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 } } peers { permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 } }
I20260808 10:38:41.957844 19851 heartbeater.cc:503] Master 127.30.146.190:32911 was elected leader, sending a full tablet report...
I20260808 10:38:41.958240 19872 ts_tablet_manager.cc:1428] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 10:38:41.978583 19564 tablet.cc:2369] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:38:42.027124 19877 raft_consensus.cc:482] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:42.027388 19877 raft_consensus.cc:504] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae88f227b4647fda2d587e7656c3d4f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43697 } } peers { permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 } } peers { permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 } }
I20260808 10:38:42.028623 19877 leader_election.cc:288] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16526aaec736492793b467eb1afa13d6 (127.30.146.130:36031), 8b5caee635da4ebaac7c7cf8d47acfa1 (127.30.146.131:34461)
I20260808 10:38:42.033877 19867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:42.029072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:39637 (local address 127.30.146.130:36031)
0808 10:38:42.029244 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:42.029248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:42.029447 (+ 199us) server_negotiation.cc:407] Connection header received
0808 10:38:42.029650 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:42.029653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:42.029714 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:42.029821 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:42.030126 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:42.030970 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:42.032604 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:42.033241 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:42.033372 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:42.033462 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:42.033663 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 10:38:42.033745 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:38:42.033916 19879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:42.028973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:36031 (local address 127.30.146.129:39637)
0808 10:38:42.029339 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:42.029352 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:42.029505 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:42.029824 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:42.029828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:42.029849 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:42.030002 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:42.030008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:42.031115 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:42.031117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:42.032459 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:42.032471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:42.033397 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:42.033401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:42.033525 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:42.033560 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 10:38:42.033643 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:38:42.033723 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260808 10:38:42.034031 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:42.029601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:54759 (local address 127.30.146.131:34461)
0808 10:38:42.029748 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:42.029752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:42.029970 (+ 218us) server_negotiation.cc:407] Connection header received
0808 10:38:42.030111 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:42.030114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:42.030162 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:42.030240 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:42.030500 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:42.031213 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:42.032605 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:42.033241 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:42.033372 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:42.033425 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:42.033788 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 10:38:42.033888 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:38:42.034231 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:42.029453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:34461 (local address 127.30.146.129:54759)
0808 10:38:42.029887 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:42.029900 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:42.029997 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:42.030263 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:42.030264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:42.030279 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:42.030387 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:42.030391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:42.031320 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:42.031324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:42.032485 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:42.032492 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:42.033568 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:42.033572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:42.033652 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:42.033671 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:38:42.033769 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:38:42.033847 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":30,"thread_start_us":144,"threads_started":1}
I20260808 10:38:42.034720 19659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5c3333c01f843c597d7fb3b92fd1afd" candidate_uuid: "8ae88f227b4647fda2d587e7656c3d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16526aaec736492793b467eb1afa13d6" is_pre_election: true
I20260808 10:38:42.034936 19802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5c3333c01f843c597d7fb3b92fd1afd" candidate_uuid: "8ae88f227b4647fda2d587e7656c3d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" is_pre_election: true
I20260808 10:38:42.035130 19659 raft_consensus.cc:2454] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ae88f227b4647fda2d587e7656c3d4f in term 0.
I20260808 10:38:42.035179 19802 raft_consensus.cc:2454] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ae88f227b4647fda2d587e7656c3d4f in term 0.
I20260808 10:38:42.035706 19442 leader_election.cc:302] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ae88f227b4647fda2d587e7656c3d4f, 8b5caee635da4ebaac7c7cf8d47acfa1; no voters:
I20260808 10:38:42.036029 19877 raft_consensus.cc:2781] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:38:42.036113 19877 raft_consensus.cc:482] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:38:42.036144 19877 raft_consensus.cc:3037] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:42.037297 19877 raft_consensus.cc:504] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae88f227b4647fda2d587e7656c3d4f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43697 } } peers { permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 } } peers { permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 } }
I20260808 10:38:42.037660 19877 leader_election.cc:288] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [CANDIDATE]: Term 1 election: Requested vote from peers 16526aaec736492793b467eb1afa13d6 (127.30.146.130:36031), 8b5caee635da4ebaac7c7cf8d47acfa1 (127.30.146.131:34461)
I20260808 10:38:42.038064 19659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5c3333c01f843c597d7fb3b92fd1afd" candidate_uuid: "8ae88f227b4647fda2d587e7656c3d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16526aaec736492793b467eb1afa13d6"
I20260808 10:38:42.038159 19802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5c3333c01f843c597d7fb3b92fd1afd" candidate_uuid: "8ae88f227b4647fda2d587e7656c3d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1"
I20260808 10:38:42.038177 19659 raft_consensus.cc:3037] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:42.038329 19802 raft_consensus.cc:3037] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:42.039335 19659 raft_consensus.cc:2454] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae88f227b4647fda2d587e7656c3d4f in term 1.
I20260808 10:38:42.039459 19802 raft_consensus.cc:2454] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae88f227b4647fda2d587e7656c3d4f in term 1.
I20260808 10:38:42.039726 19442 leader_election.cc:302] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16526aaec736492793b467eb1afa13d6, 8ae88f227b4647fda2d587e7656c3d4f; no voters:
I20260808 10:38:42.039945 19877 raft_consensus.cc:2781] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:42.040205 19877 raft_consensus.cc:686] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [term 1 LEADER]: Becoming Leader. State: Replica: 8ae88f227b4647fda2d587e7656c3d4f, State: Running, Role: LEADER
I20260808 10:38:42.040555 19877 consensus_queue.cc:237] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae88f227b4647fda2d587e7656c3d4f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43697 } } peers { permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 } } peers { permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 } }
I20260808 10:38:42.044044 19356 catalog_manager.cc:5495] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ae88f227b4647fda2d587e7656c3d4f (127.30.146.129). New cstate: current_term: 1 leader_uuid: "8ae88f227b4647fda2d587e7656c3d4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae88f227b4647fda2d587e7656c3d4f" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 43697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260808 10:38:42.080559 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:42.076857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34706 (local address 127.30.146.129:43697)
0808 10:38:42.077033 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:42.077039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:42.077061 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:38:42.077132 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:42.077134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:42.077247 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:42.077368 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:38:42.077631 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:42.078150 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:42.079034 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:42.079254 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:42.079368 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:42.079683 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:38:42.080088 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:38:42.080257 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:42.080365 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 10:38:42.080404 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:38:42.088706 19659 raft_consensus.cc:1261] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6 [term 1 FOLLOWER]: Refusing update from remote peer 8ae88f227b4647fda2d587e7656c3d4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:42.088896 19802 raft_consensus.cc:1261] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1 [term 1 FOLLOWER]: Refusing update from remote peer 8ae88f227b4647fda2d587e7656c3d4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:42.089597 19881 consensus_queue.cc:1035] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "16526aaec736492793b467eb1afa13d6" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 36031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:42.089844 19877 consensus_queue.cc:1035] T f5c3333c01f843c597d7fb3b92fd1afd P 8ae88f227b4647fda2d587e7656c3d4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5caee635da4ebaac7c7cf8d47acfa1" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:42.100771 19887 mvcc.cc:205] Tried to move back new op lower bound from 7316215898467176448 to 7316215898281672704. Current Snapshot: MvccSnapshot[applied={T|T < 7316215898467176448}]
I20260808 10:38:42.107450 19356 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:42.107652 19356 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:42.110302 19356 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 171333f98801477b99b66c2b18931d71: Sending DeleteTablet for 3 replicas of tablet f5c3333c01f843c597d7fb3b92fd1afd
I20260808 10:38:42.111091 19493 tablet_service.cc:1514] Processing DeleteTablet for tablet f5c3333c01f843c597d7fb3b92fd1afd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:42 UTC) from {username='slave'} at 127.0.0.1:34692
I20260808 10:38:42.111241 19781 tablet_service.cc:1514] Processing DeleteTablet for tablet f5c3333c01f843c597d7fb3b92fd1afd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:42 UTC) from {username='slave'} at 127.0.0.1:52046
I20260808 10:38:42.111701 19635 tablet_service.cc:1514] Processing DeleteTablet for tablet f5c3333c01f843c597d7fb3b92fd1afd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:42 UTC) from {username='slave'} at 127.0.0.1:50840
I20260808 10:38:42.112206 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 19425
I20260808 10:38:42.114897 19897 tablet_replica.cc:330] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: stopping tablet replica
I20260808 10:38:42.115350 19897 raft_consensus.cc:2229] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:38:42.115423 19898 tablet_replica.cc:330] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: stopping tablet replica
I20260808 10:38:42.115720 19897 raft_consensus.cc:2258] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:42.115737 19898 raft_consensus.cc:2229] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:38:42.116065 19898 raft_consensus.cc:2258] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:42.117432 19898 ts_tablet_manager.cc:1905] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:42.117470 19897 ts_tablet_manager.cc:1905] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:42.119969 19898 ts_tablet_manager.cc:1918] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:42.120095 19898 log.cc:1196] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/f5c3333c01f843c597d7fb3b92fd1afd
I20260808 10:38:42.120445 19898 ts_tablet_manager.cc:1939] T f5c3333c01f843c597d7fb3b92fd1afd P 16526aaec736492793b467eb1afa13d6: Deleting consensus metadata
I20260808 10:38:42.121089 19897 ts_tablet_manager.cc:1918] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:42.121220 19897 log.cc:1196] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/f5c3333c01f843c597d7fb3b92fd1afd
I20260808 10:38:42.121551 19897 ts_tablet_manager.cc:1939] T f5c3333c01f843c597d7fb3b92fd1afd P 8b5caee635da4ebaac7c7cf8d47acfa1: Deleting consensus metadata
I20260808 10:38:42.121809 19342 catalog_manager.cc:4841] TS 16526aaec736492793b467eb1afa13d6 (127.30.146.130:36031): tablet f5c3333c01f843c597d7fb3b92fd1afd (table test-table [id=924f57281d39431ba8588756331ee561]) successfully deleted
I20260808 10:38:42.122812 19342 catalog_manager.cc:4841] TS 8b5caee635da4ebaac7c7cf8d47acfa1 (127.30.146.131:34461): tablet f5c3333c01f843c597d7fb3b92fd1afd (table test-table [id=924f57281d39431ba8588756331ee561]) successfully deleted
W20260808 10:38:42.123383 19342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:43697 (error 108)
I20260808 10:38:42.123589 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 19566
I20260808 10:38:42.130362 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:42.129912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:43697 (local address 127.0.0.1:34722)
0808 10:38:42.130117 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:42.130231 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 10:38:42.130596 19342 catalog_manager.cc:4584] TS 8ae88f227b4647fda2d587e7656c3d4f (127.30.146.129:43697): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f5c3333c01f843c597d7fb3b92fd1afd: Network error: Client connection negotiation failed: client connection to 127.30.146.129:43697: connect: Connection refused (error 111)
I20260808 10:38:42.132413 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 19712
I20260808 10:38:42.140483 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 19324
2026-08-08T10:38:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1155 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:42 dist-test-slave-9gcw krb5kdc[19902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:42 dist-test-slave-9gcw krb5kdc[19902](info): set up 2 sockets
Aug 08 10:38:42 dist-test-slave-9gcw krb5kdc[19902](info): commencing operation
krb5kdc: starting...
W20260808 10:38:43.263900 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:43 dist-test-slave-9gcw krb5kdc[19902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:38:43.401775 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:34765
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:40357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:34765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:43.521394 19918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:43.521710 19918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:43.521788 19918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:43.525972 19918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:43.526064 19918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:43.526103 19918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:43.526165 19918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:43.526203 19918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:43.532483 19918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:40357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:34765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:34765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:43.533737 19918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:43.534817 19918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:43.543864 19926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:43.543968 19918 server_base.cc:1034] running on GCE node
W20260808 10:38:43.543934 19924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:43.543864 19923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:43.544791 19918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:43.545903 19918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:43.547067 19918 hybrid_clock.cc:648] HybridClock initialized: now 1786185523547033 us; error 47 us; skew 500 ppm
Aug 08 10:38:43 dist-test-slave-9gcw krb5kdc[19902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:43.550184 19918 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:38:43.551167 19918 webserver.cc:460] Webserver started at http://127.30.146.190:44323/ using document root <none> and password file <none>
I20260808 10:38:43.551735 19918 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:43.551798 19918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:43.552052 19918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:43.554278 19918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "fed7f16c8c2c48678e5924b096d122c3"
format_stamp: "Formatted at 2026-08-08 10:38:43 on dist-test-slave-9gcw"
server_key: "67b6825de73d68f6d9950472bc789ca2"
server_key_iv: "3629fbf00703a77b438cdfce04be35b2"
server_key_version: "encryptionkey@0"
I20260808 10:38:43.554888 19918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "fed7f16c8c2c48678e5924b096d122c3"
format_stamp: "Formatted at 2026-08-08 10:38:43 on dist-test-slave-9gcw"
server_key: "67b6825de73d68f6d9950472bc789ca2"
server_key_iv: "3629fbf00703a77b438cdfce04be35b2"
server_key_version: "encryptionkey@0"
I20260808 10:38:43.558844 19918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 10:38:43.561367 19934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:43.562682 19918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:38:43.562848 19918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "fed7f16c8c2c48678e5924b096d122c3"
format_stamp: "Formatted at 2026-08-08 10:38:43 on dist-test-slave-9gcw"
server_key: "67b6825de73d68f6d9950472bc789ca2"
server_key_iv: "3629fbf00703a77b438cdfce04be35b2"
server_key_version: "encryptionkey@0"
I20260808 10:38:43.562968 19918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:43.581308 19918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:43.584699 19918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:43.584914 19918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:43.592248 19918 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:34765
I20260808 10:38:43.592268 19989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:34765 every 8 connection(s)
I20260808 10:38:43.593251 19918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:43.596019 19990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:43.599565 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 19918
I20260808 10:38:43.599730 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:43.600024 31306 external_mini_cluster.cc:1428] Setting key 4d9ca877cd1742dcf3bf2e589652b688
I20260808 10:38:43.602072 19990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3: Bootstrap starting.
I20260808 10:38:43.604842 19990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:43.605777 19990 log.cc:824] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:43.608070 19990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3: No bootstrap required, opened a new log
Aug 08 10:38:43 dist-test-slave-9gcw krb5kdc[19902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:43.612727 19990 raft_consensus.cc:348] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7f16c8c2c48678e5924b096d122c3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34765 } }
I20260808 10:38:43.613049 19990 raft_consensus.cc:374] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:43.613111 19990 raft_consensus.cc:729] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed7f16c8c2c48678e5924b096d122c3, State: Initialized, Role: FOLLOWER
I20260808 10:38:43.613626 19990 consensus_queue.cc:260] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7f16c8c2c48678e5924b096d122c3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34765 } }
I20260808 10:38:43.613814 19990 raft_consensus.cc:388] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:43.613901 19990 raft_consensus.cc:482] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:43.614035 19990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:43.615286 19990 raft_consensus.cc:504] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7f16c8c2c48678e5924b096d122c3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34765 } }
I20260808 10:38:43.615677 19990 leader_election.cc:302] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fed7f16c8c2c48678e5924b096d122c3; no voters:
I20260808 10:38:43.616021 19990 leader_election.cc:288] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:43.616128 19995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:43.616386 19995 raft_consensus.cc:686] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [term 1 LEADER]: Becoming Leader. State: Replica: fed7f16c8c2c48678e5924b096d122c3, State: Running, Role: LEADER
I20260808 10:38:43.616621 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:43.601660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.30.146.190:34765)
0808 10:38:43.602123 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:43.602145 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:43.602189 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:38:43.602992 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:43.603014 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:43.603453 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:43.603809 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:43.604552 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:43.606113 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:43.607192 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:43.607875 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:43.608035 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:43.610946 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:43.610954 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:43.610979 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:43.611020 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:43.613091 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:43.613653 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:43.613659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:43.613665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:43.613816 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:43.614182 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:43.614186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:43.614187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:43.614736 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:43.614957 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:43.615165 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:38:43.615553 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":114,"threads_started":1}
I20260808 10:38:43.616705 19995 consensus_queue.cc:237] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7f16c8c2c48678e5924b096d122c3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34765 } }
I20260808 10:38:43.617338 19990 sys_catalog.cc:564] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:43.618508 19996 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fed7f16c8c2c48678e5924b096d122c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7f16c8c2c48678e5924b096d122c3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34765 } } }
I20260808 10:38:43.618623 19996 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:43.619016 19997 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fed7f16c8c2c48678e5924b096d122c3. Latest consensus state: current_term: 1 leader_uuid: "fed7f16c8c2c48678e5924b096d122c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7f16c8c2c48678e5924b096d122c3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34765 } } }
I20260808 10:38:43.619123 19997 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3 [sys.catalog]: This master's current role is: LEADER
W20260808 10:38:43.622442 20012 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:38:43.622540 20012 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:38:43.622614 20006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:43.623356 20006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:43.628693 20006 catalog_manager.cc:1347] Generated new cluster ID: f36335826eb64007a1942c749afec57c
I20260808 10:38:43.628759 20006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:43.647377 20006 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:43.648738 20006 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:43.659207 20006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3: Generated new TSK 0
I20260808 10:38:43.660141 20006 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
Entry for principal kudu/127.30.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab.
I20260808 10:38:43.697860 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34765
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:40357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:43.815107 20018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:43.815387 20018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:43.815436 20018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:43.819670 20018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:43.819728 20018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:43.819785 20018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:38:43.825258 20018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:40357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.129.keytab
--principal=kudu/127.30.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:43.826268 20018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:43.827255 20018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:43.837064 20026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:43.837193 20023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:43.837064 20024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:43.837306 20018 server_base.cc:1034] running on GCE node
I20260808 10:38:43.837786 20018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:43.838382 20018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:43.839545 20018 hybrid_clock.cc:648] HybridClock initialized: now 1786185523839513 us; error 44 us; skew 500 ppm
Aug 08 10:38:43 dist-test-slave-9gcw krb5kdc[19902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:43.842672 20018 init.cc:377] Logged in from keytab as kudu/127.30.146.129@KRBTEST.COM (short username kudu)
I20260808 10:38:43.843523 20018 webserver.cc:460] Webserver started at http://127.30.146.129:46603/ using document root <none> and password file <none>
I20260808 10:38:43.844049 20018 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:43.844107 20018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:43.844286 20018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:43.846493 20018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "a96a00aeabba45f48615ed11571bc4c3"
format_stamp: "Formatted at 2026-08-08 10:38:43 on dist-test-slave-9gcw"
server_key: "fcf4280507d7de3585672f2db7fb7637"
server_key_iv: "e632921a8a39a6d93b1ad3fd2514e529"
server_key_version: "encryptionkey@0"
I20260808 10:38:43.847007 20018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "a96a00aeabba45f48615ed11571bc4c3"
format_stamp: "Formatted at 2026-08-08 10:38:43 on dist-test-slave-9gcw"
server_key: "fcf4280507d7de3585672f2db7fb7637"
server_key_iv: "e632921a8a39a6d93b1ad3fd2514e529"
server_key_version: "encryptionkey@0"
I20260808 10:38:43.850929 20018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:38:43.853556 20035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:43.855086 20018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:38:43.855258 20018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "a96a00aeabba45f48615ed11571bc4c3"
format_stamp: "Formatted at 2026-08-08 10:38:43 on dist-test-slave-9gcw"
server_key: "fcf4280507d7de3585672f2db7fb7637"
server_key_iv: "e632921a8a39a6d93b1ad3fd2514e529"
server_key_version: "encryptionkey@0"
I20260808 10:38:43.855396 20018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:43.870509 20018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:43.874166 20018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:43.874395 20018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:43.875067 20018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:43.875983 20018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:43.876035 20018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:43.876123 20018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:43.876163 20018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:43.886512 20018 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:34927
I20260808 10:38:43.886536 20157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:34927 every 8 connection(s)
I20260808 10:38:43.887478 20018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:38:43.895651 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 20018
I20260808 10:38:43.895810 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:38:43.896168 31306 external_mini_cluster.cc:1428] Setting key d6de022f2dfdf41faf4d05079dd15c1d
Aug 08 10:38:43 dist-test-slave-9gcw krb5kdc[19902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185523, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.129@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:43.902325 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:43.889418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:47805 (local address 127.30.146.190:34765)
0808 10:38:43.889577 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:43.889583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:43.890495 (+ 912us) server_negotiation.cc:407] Connection header received
0808 10:38:43.891939 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:43.891941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:43.891990 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:43.892067 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:43.893511 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:43.894052 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:43.895351 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:43.895576 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:43.895660 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:43.898496 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:43.898500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:43.898502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:43.898533 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:43.900091 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:43.900632 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:43.900640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:43.900641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:43.900725 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:43.901111 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:43.901114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:43.901116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:43.901358 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:43.901468 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:43.902034 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 10:38:43.902162 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:38:43.903239 20161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:43.889850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34765 (local address 127.30.146.129:47805)
0808 10:38:43.890335 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:43.890381 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:43.891730 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:43.892285 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:43.892292 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:43.892736 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:43.893361 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:43.893379 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:43.894299 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:43.894302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:43.895217 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:43.895227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:43.895744 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:43.895750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:43.895983 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:43.896619 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:43.896645 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:43.898333 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:43.900226 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:43.900231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:43.900245 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:43.900517 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:43.900824 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:43.900827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:43.900829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:43.901002 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:43.901467 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:43.901474 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:43.901786 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 10:38:43.901976 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 10:38:43.902327 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":112,"threads_started":1}
I20260808 10:38:43.904628 20158 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34765
I20260808 10:38:43.904920 20158 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:43.905488 20158 heartbeater.cc:511] Master 127.30.146.190:34765 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.146.130@KRBTEST.COM; defaulting to no policy
I20260808 10:38:43.907148 19952 ts_manager.cc:194] Registered new tserver with Master: a96a00aeabba45f48615ed11571bc4c3 (127.30.146.129:34927)
I20260808 10:38:43.908432 19952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.129@KRBTEST.COM'} at 127.30.146.129:47805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
Entry for principal kudu/127.30.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab.
I20260808 10:38:43.929039 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34765
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:40357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:44.049985 20164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:44.050267 20164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:44.050313 20164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:44.054988 20164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:44.055071 20164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:44.055142 20164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:38:44.062320 20164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:40357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.130.keytab
--principal=kudu/127.30.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:44.063488 20164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:44.064541 20164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:44.075147 20170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:44.075179 20169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:44.075269 20164 server_base.cc:1034] running on GCE node
W20260808 10:38:44.075157 20172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:44.075964 20164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:44.076706 20164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:44.077895 20164 hybrid_clock.cc:648] HybridClock initialized: now 1786185524077882 us; error 25 us; skew 500 ppm
Aug 08 10:38:44 dist-test-slave-9gcw krb5kdc[19902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185524, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:44.080864 20164 init.cc:377] Logged in from keytab as kudu/127.30.146.130@KRBTEST.COM (short username kudu)
I20260808 10:38:44.081837 20164 webserver.cc:460] Webserver started at http://127.30.146.130:37843/ using document root <none> and password file <none>
I20260808 10:38:44.082423 20164 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:44.082520 20164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:44.082762 20164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:44.084903 20164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "bb343de1be3245a6867b8f1a4f0d60ec"
format_stamp: "Formatted at 2026-08-08 10:38:44 on dist-test-slave-9gcw"
server_key: "f1fa0d53663684288af9157c8345682a"
server_key_iv: "e09b8df7ea61306665248f39c6a10c18"
server_key_version: "encryptionkey@0"
I20260808 10:38:44.085543 20164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "bb343de1be3245a6867b8f1a4f0d60ec"
format_stamp: "Formatted at 2026-08-08 10:38:44 on dist-test-slave-9gcw"
server_key: "f1fa0d53663684288af9157c8345682a"
server_key_iv: "e09b8df7ea61306665248f39c6a10c18"
server_key_version: "encryptionkey@0"
I20260808 10:38:44.089684 20164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:38:44.092185 20181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:44.093537 20164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:38:44.093688 20164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "bb343de1be3245a6867b8f1a4f0d60ec"
format_stamp: "Formatted at 2026-08-08 10:38:44 on dist-test-slave-9gcw"
server_key: "f1fa0d53663684288af9157c8345682a"
server_key_iv: "e09b8df7ea61306665248f39c6a10c18"
server_key_version: "encryptionkey@0"
I20260808 10:38:44.093803 20164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:44.124202 20164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:44.127835 20164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:44.128069 20164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:44.128715 20164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:44.129946 20164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:44.130029 20164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:44.130105 20164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:44.130146 20164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:44.139962 20164 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:41101
I20260808 10:38:44.139999 20303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:41101 every 8 connection(s)
I20260808 10:38:44.140933 20164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:38:44.147202 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 20164
I20260808 10:38:44.147364 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:38:44.147660 31306 external_mini_cluster.cc:1428] Setting key dbd027794c1cae02a0d33f56a96f4200
Aug 08 10:38:44 dist-test-slave-9gcw krb5kdc[19902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185524, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.130@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:44.155962 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.142880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:35451 (local address 127.30.146.190:34765)
0808 10:38:44.143015 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:44.143019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:44.143819 (+ 800us) server_negotiation.cc:407] Connection header received
0808 10:38:44.145138 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:44.145141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:44.145226 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:44.145301 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:44.146546 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.147107 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.148519 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.148852 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.148955 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.151625 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:44.151630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:44.151635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:44.151670 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:44.153292 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:44.153908 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:44.153910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:44.153911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:44.153989 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:44.154490 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:44.154492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:44.154493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:44.154831 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:44.154916 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:44.155639 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 10:38:44.155784 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:38:44.156793 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.143223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34765 (local address 127.30.146.130:35451)
0808 10:38:44.143661 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:44.143709 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:44.144932 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:44.145477 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:44.145483 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:44.145865 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:44.146382 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.146397 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.147254 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.147257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.148413 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.148425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.149020 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.149025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.149257 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.149974 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:44.149991 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:44.151444 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:44.153403 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:44.153412 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:44.153430 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:44.153800 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:44.154089 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:44.154095 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:44.154140 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:44.154372 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:44.154956 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:44.154963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:44.155353 (+ 390us) client_negotiation.cc:769] Sending connection context
0808 10:38:44.155593 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 10:38:44.155927 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":101,"threads_started":1}
I20260808 10:38:44.158022 20304 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34765
I20260808 10:38:44.158282 20304 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:44.158787 20304 heartbeater.cc:511] Master 127.30.146.190:34765 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.146.131@KRBTEST.COM; defaulting to no policy
I20260808 10:38:44.160080 19952 ts_manager.cc:194] Registered new tserver with Master: bb343de1be3245a6867b8f1a4f0d60ec (127.30.146.130:41101)
I20260808 10:38:44.160933 19952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.130@KRBTEST.COM'} at 127.30.146.130:35451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
Entry for principal kudu/127.30.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab.
I20260808 10:38:44.180619 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34765
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:40357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:44.300993 20309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:44.301302 20309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:44.301390 20309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:44.305536 20309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:44.305595 20309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:44.305693 20309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:38:44.311177 20309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:40357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.131.keytab
--principal=kudu/127.30.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:34765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:44.312322 20309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:44.313393 20309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:44.323649 20316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:44.323657 20318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:44.323709 20309 server_base.cc:1034] running on GCE node
W20260808 10:38:44.323664 20315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:44.324236 20309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:44.324883 20309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:44.326049 20309 hybrid_clock.cc:648] HybridClock initialized: now 1786185524326019 us; error 42 us; skew 500 ppm
Aug 08 10:38:44 dist-test-slave-9gcw krb5kdc[19902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185524, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:44.328933 20309 init.cc:377] Logged in from keytab as kudu/127.30.146.131@KRBTEST.COM (short username kudu)
I20260808 10:38:44.329934 20309 webserver.cc:460] Webserver started at http://127.30.146.131:32881/ using document root <none> and password file <none>
I20260808 10:38:44.330508 20309 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:44.330603 20309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:44.330845 20309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:44.332870 20309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "fb21091e2bf04cf281142cdcbd97f59c"
format_stamp: "Formatted at 2026-08-08 10:38:44 on dist-test-slave-9gcw"
server_key: "d249932b10539e2eb6e538690d36b89d"
server_key_iv: "a8bdaa6303b087775bd9e32094249598"
server_key_version: "encryptionkey@0"
I20260808 10:38:44.333461 20309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "fb21091e2bf04cf281142cdcbd97f59c"
format_stamp: "Formatted at 2026-08-08 10:38:44 on dist-test-slave-9gcw"
server_key: "d249932b10539e2eb6e538690d36b89d"
server_key_iv: "a8bdaa6303b087775bd9e32094249598"
server_key_version: "encryptionkey@0"
I20260808 10:38:44.337231 20309 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:38:44.339669 20325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:44.340972 20309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:38:44.341118 20309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "fb21091e2bf04cf281142cdcbd97f59c"
format_stamp: "Formatted at 2026-08-08 10:38:44 on dist-test-slave-9gcw"
server_key: "d249932b10539e2eb6e538690d36b89d"
server_key_iv: "a8bdaa6303b087775bd9e32094249598"
server_key_version: "encryptionkey@0"
I20260808 10:38:44.341280 20309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:44.356606 20309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:44.359760 20309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:44.359977 20309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:44.360594 20309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:44.361606 20309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:44.361656 20309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:44.361697 20309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:44.361712 20309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:44.371387 20309 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:45067
I20260808 10:38:44.371420 20447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:45067 every 8 connection(s)
I20260808 10:38:44.372309 20309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:38:44.378324 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 20309
I20260808 10:38:44.378436 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:38:44.378750 31306 external_mini_cluster.cc:1428] Setting key f863b9013a79b4049ccf1243271c92b7
Aug 08 10:38:44 dist-test-slave-9gcw krb5kdc[19902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185524, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.131@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:38:44.387876 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.374098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:52123 (local address 127.30.146.190:34765)
0808 10:38:44.374259 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:44.374265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:44.375225 (+ 960us) server_negotiation.cc:407] Connection header received
0808 10:38:44.376544 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:44.376547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:44.376595 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:44.376671 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:44.378011 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.378859 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.380783 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.381063 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.381200 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.384212 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:44.384214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:44.384217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:44.384240 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:44.385846 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:44.386393 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:44.386395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:44.386396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:44.386470 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:44.386868 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:44.386871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:44.386872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:44.387047 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:44.387123 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:44.387635 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 10:38:44.387743 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:38:44.388902 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.374588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:34765 (local address 127.30.146.131:52123)
0808 10:38:44.375056 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:44.375110 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:44.376340 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:44.376837 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:44.376844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:44.377287 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:44.377835 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.377850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.379036 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.379046 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.380649 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.380662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.381270 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.381277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.381508 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.382288 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:38:44.382308 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:44.384042 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:44.385966 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:44.385971 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:44.385983 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:44.386296 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:44.386554 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:38:44.386557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:38:44.386558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:38:44.386776 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:38:44.387148 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:44.387154 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:44.387410 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 10:38:44.387587 (+ 177us) client_negotiation.cc:240] Negotiation successful
0808 10:38:44.387992 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":117,"threads_started":1}
I20260808 10:38:44.390205 20448 heartbeater.cc:347] Connected to a master server at 127.30.146.190:34765
I20260808 10:38:44.390470 20448 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:44.391016 20448 heartbeater.cc:511] Master 127.30.146.190:34765 requested a full tablet report, sending...
I20260808 10:38:44.392333 19950 ts_manager.cc:194] Registered new tserver with Master: fb21091e2bf04cf281142cdcbd97f59c (127.30.146.131:45067)
I20260808 10:38:44.393316 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:38:44.393307 19950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.146.131@KRBTEST.COM'} at 127.30.146.131:52123
I20260808 10:38:44.403162 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.395672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.30.146.190:34765)
0808 10:38:44.395806 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:44.395809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:44.395976 (+ 167us) server_negotiation.cc:407] Connection header received
0808 10:38:44.396078 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:44.396080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:44.396142 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:44.396215 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:44.396981 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.397625 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.398628 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.398850 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.398933 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.400185 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:44.400188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:44.400221 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:44.400243 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:44.401685 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:44.402068 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:44.402071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:44.402072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:44.402148 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:44.402417 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:44.402420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:44.402421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:44.402622 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:44.402727 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:44.402956 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 10:38:44.403047 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:38:44.406246 19950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:38:44.409325 19950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:38:44.425221 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.418937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:45067 (local address 127.0.0.1:40382)
0808 10:38:44.419915 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:44.419929 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:44.420024 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:44.421603 (+ 1579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:44.421608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:44.421629 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:44.421720 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.421728 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.422901 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.422907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.423750 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.423759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.424606 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.424610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.424717 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.424782 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 10:38:44.424904 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 10:38:44.425035 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"mutex_wait_us":39,"thread_start_us":109,"threads_started":1}
I20260808 10:38:44.425561 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.417964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34927 (local address 127.0.0.1:45580)
0808 10:38:44.419110 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:44.419162 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:44.419309 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:44.420638 (+ 1329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:44.420646 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:44.420684 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:44.420823 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.420830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.422290 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.422293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.423809 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.423819 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.425114 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.425117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.425231 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.425254 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:38:44.425335 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 10:38:44.425412 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"mutex_wait_us":86,"spinlock_wait_cycles":3328,"thread_start_us":190,"threads_started":1}
I20260808 10:38:44.425799 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.418691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40382 (local address 127.30.146.131:45067)
0808 10:38:44.420930 (+ 2239us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:44.420941 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:44.420965 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:38:44.421047 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:44.421054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:44.421260 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:44.421429 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:44.421857 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.422765 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.423881 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.424473 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.424607 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.424748 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:44.425537 (+ 789us) server_negotiation.cc:299] Negotiation successful
0808 10:38:44.425648 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2134,"thread_start_us":284,"threads_started":1}
I20260808 10:38:44.425830 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.419678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:41101 (local address 127.0.0.1:39496)
0808 10:38:44.420556 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:44.420570 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:44.420670 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:44.421317 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:44.421320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:44.421338 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:44.421431 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.421437 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.422821 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.422824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.424019 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.424027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.425207 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.425210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.425287 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.425320 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:38:44.425375 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:38:44.425420 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":53,"thread_start_us":99,"threads_started":1}
I20260808 10:38:44.425827 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.420116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39496 (local address 127.30.146.130:41101)
0808 10:38:44.420527 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:44.420539 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:44.420662 (+ 123us) server_negotiation.cc:407] Connection header received
0808 10:38:44.420774 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:44.420789 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:44.420984 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:44.421130 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:44.421554 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.422657 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.424142 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.424962 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.425072 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.425517 (+ 445us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:44.425569 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 10:38:44.425669 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":154,"threads_started":1}
I20260808 10:38:44.426962 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.418119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.30.146.129:34927)
0808 10:38:44.418614 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:44.418621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:44.419950 (+ 1329us) server_negotiation.cc:407] Connection header received
0808 10:38:44.420050 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:44.420056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:44.420276 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:44.420448 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:44.420964 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.422145 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.423944 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.424793 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.424918 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.425048 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:44.426728 (+ 1680us) server_negotiation.cc:299] Negotiation successful
0808 10:38:44.426825 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":105,"threads_started":1}
I20260808 10:38:44.428378 20234 tablet_service.cc:1467] Processing CreateTablet for tablet 32c23364ae5e498f8e377e95466afcb5 (DEFAULT_TABLE table=test-table [id=d159aa48f991494ba0732d5642766fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:44.428375 20377 tablet_service.cc:1467] Processing CreateTablet for tablet 32c23364ae5e498f8e377e95466afcb5 (DEFAULT_TABLE table=test-table [id=d159aa48f991494ba0732d5642766fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:44.429418 20234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32c23364ae5e498f8e377e95466afcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:44.429428 20377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32c23364ae5e498f8e377e95466afcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:44.431519 20088 tablet_service.cc:1467] Processing CreateTablet for tablet 32c23364ae5e498f8e377e95466afcb5 (DEFAULT_TABLE table=test-table [id=d159aa48f991494ba0732d5642766fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:44.432590 20088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32c23364ae5e498f8e377e95466afcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:44.435863 20466 tablet_bootstrap.cc:492] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: Bootstrap starting.
I20260808 10:38:44.437105 20467 tablet_bootstrap.cc:492] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: Bootstrap starting.
I20260808 10:38:44.438531 20466 tablet_bootstrap.cc:654] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:44.439275 20467 tablet_bootstrap.cc:654] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:44.439581 20468 tablet_bootstrap.cc:492] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3: Bootstrap starting.
I20260808 10:38:44.439779 20466 log.cc:824] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:44.440495 20467 log.cc:824] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:44.441869 20468 tablet_bootstrap.cc:654] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:44.442094 20466 tablet_bootstrap.cc:492] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: No bootstrap required, opened a new log
I20260808 10:38:44.442257 20467 tablet_bootstrap.cc:492] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: No bootstrap required, opened a new log
I20260808 10:38:44.442296 20466 ts_tablet_manager.cc:1397] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:38:44.442448 20467 ts_tablet_manager.cc:1397] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 10:38:44.442885 20468 log.cc:824] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:44.444916 20468 tablet_bootstrap.cc:492] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3: No bootstrap required, opened a new log
I20260808 10:38:44.445086 20468 ts_tablet_manager.cc:1397] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:38:44.445851 20466 raft_consensus.cc:348] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 } } peers { permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 } } peers { permanent_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41101 } }
I20260808 10:38:44.446130 20466 raft_consensus.cc:374] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:44.446187 20466 raft_consensus.cc:729] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb343de1be3245a6867b8f1a4f0d60ec, State: Initialized, Role: FOLLOWER
I20260808 10:38:44.446414 20467 raft_consensus.cc:348] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 } } peers { permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 } } peers { permanent_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41101 } }
I20260808 10:38:44.446653 20467 raft_consensus.cc:374] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:44.446722 20467 raft_consensus.cc:729] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb21091e2bf04cf281142cdcbd97f59c, State: Initialized, Role: FOLLOWER
I20260808 10:38:44.446656 20466 consensus_queue.cc:260] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 } } peers { permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 } } peers { permanent_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41101 } }
I20260808 10:38:44.447229 20467 consensus_queue.cc:260] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 } } peers { permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 } } peers { permanent_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41101 } }
I20260808 10:38:44.447350 20304 heartbeater.cc:503] Master 127.30.146.190:34765 was elected leader, sending a full tablet report...
I20260808 10:38:44.447662 20466 ts_tablet_manager.cc:1428] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:38:44.448167 20467 ts_tablet_manager.cc:1428] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 10:38:44.448246 20448 heartbeater.cc:503] Master 127.30.146.190:34765 was elected leader, sending a full tablet report...
I20260808 10:38:44.449362 20468 raft_consensus.cc:348] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 } } peers { permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 } } peers { permanent_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41101 } }
I20260808 10:38:44.449618 20468 raft_consensus.cc:374] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:44.449676 20468 raft_consensus.cc:729] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96a00aeabba45f48615ed11571bc4c3, State: Initialized, Role: FOLLOWER
I20260808 10:38:44.450089 20468 consensus_queue.cc:260] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 } } peers { permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 } } peers { permanent_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41101 } }
I20260808 10:38:44.451056 20468 ts_tablet_manager.cc:1428] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:38:44.451088 20158 heartbeater.cc:503] Master 127.30.146.190:34765 was elected leader, sending a full tablet report...
I20260808 10:38:44.503793 20472 raft_consensus.cc:482] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:44.504048 20472 raft_consensus.cc:504] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 } } peers { permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 } } peers { permanent_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41101 } }
I20260808 10:38:44.505327 20472 leader_election.cc:288] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a96a00aeabba45f48615ed11571bc4c3 (127.30.146.129:34927), fb21091e2bf04cf281142cdcbd97f59c (127.30.146.131:45067)
I20260808 10:38:44.509022 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.505648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:55521 (local address 127.30.146.129:34927)
0808 10:38:44.505794 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:44.505801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:44.505822 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:38:44.505887 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:44.505890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:44.505969 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:44.506081 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:44.506376 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.506887 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.507956 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.508392 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.508528 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.508586 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:44.508773 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 10:38:44.508851 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:38:44.509004 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.505515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34927 (local address 127.30.146.130:55521)
0808 10:38:44.505655 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:44.505674 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:44.505772 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:44.506094 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:44.506098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:44.506117 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:44.506251 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.506257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.507007 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.507011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.507822 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.507831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.508522 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.508526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.508639 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.508658 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:38:44.508755 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 10:38:44.508845 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":4}
I20260808 10:38:44.509202 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.505844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:39495 (local address 127.30.146.131:45067)
0808 10:38:44.505981 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:44.505985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:44.506084 (+ 99us) server_negotiation.cc:407] Connection header received
0808 10:38:44.506194 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:44.506197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:44.506250 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:44.506341 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:44.506645 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.507140 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.508179 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.508581 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.508713 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.508764 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:44.508954 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 10:38:44.509025 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:38:44.509554 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.505617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:45067 (local address 127.30.146.130:39495)
0808 10:38:44.505986 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:44.506000 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:44.506101 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:44.506358 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:44.506360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:44.506378 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:44.506528 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.506533 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.507261 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.507265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.508043 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:44.508052 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.508708 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.508712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:44.508813 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.508836 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:38:44.508935 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 10:38:44.509005 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":198,"threads_started":1}
I20260808 10:38:44.509811 20109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32c23364ae5e498f8e377e95466afcb5" candidate_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96a00aeabba45f48615ed11571bc4c3" is_pre_election: true
I20260808 10:38:44.510185 20109 raft_consensus.cc:2454] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb343de1be3245a6867b8f1a4f0d60ec in term 0.
I20260808 10:38:44.510280 20398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32c23364ae5e498f8e377e95466afcb5" candidate_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb21091e2bf04cf281142cdcbd97f59c" is_pre_election: true
I20260808 10:38:44.510571 20398 raft_consensus.cc:2454] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb343de1be3245a6867b8f1a4f0d60ec in term 0.
I20260808 10:38:44.510792 20185 leader_election.cc:302] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a96a00aeabba45f48615ed11571bc4c3, bb343de1be3245a6867b8f1a4f0d60ec; no voters:
I20260808 10:38:44.511160 20472 raft_consensus.cc:2781] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:38:44.511250 20472 raft_consensus.cc:482] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:38:44.511281 20472 raft_consensus.cc:3037] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:44.512430 20472 raft_consensus.cc:504] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 } } peers { permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 } } peers { permanent_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41101 } }
I20260808 10:38:44.512816 20472 leader_election.cc:288] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [CANDIDATE]: Term 1 election: Requested vote from peers a96a00aeabba45f48615ed11571bc4c3 (127.30.146.129:34927), fb21091e2bf04cf281142cdcbd97f59c (127.30.146.131:45067)
I20260808 10:38:44.513230 20109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32c23364ae5e498f8e377e95466afcb5" candidate_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96a00aeabba45f48615ed11571bc4c3"
I20260808 10:38:44.513252 20398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32c23364ae5e498f8e377e95466afcb5" candidate_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb21091e2bf04cf281142cdcbd97f59c"
I20260808 10:38:44.513357 20109 raft_consensus.cc:3037] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:44.513386 20398 raft_consensus.cc:3037] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:44.514506 20398 raft_consensus.cc:2454] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb343de1be3245a6867b8f1a4f0d60ec in term 1.
I20260808 10:38:44.514581 20109 raft_consensus.cc:2454] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb343de1be3245a6867b8f1a4f0d60ec in term 1.
I20260808 10:38:44.514876 20184 leader_election.cc:302] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb343de1be3245a6867b8f1a4f0d60ec, fb21091e2bf04cf281142cdcbd97f59c; no voters:
I20260808 10:38:44.515131 20472 raft_consensus.cc:2781] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:44.515388 20472 raft_consensus.cc:686] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 1 LEADER]: Becoming Leader. State: Replica: bb343de1be3245a6867b8f1a4f0d60ec, State: Running, Role: LEADER
I20260808 10:38:44.515727 20472 consensus_queue.cc:237] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 } } peers { permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 } } peers { permanent_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41101 } }
I20260808 10:38:44.519237 19952 catalog_manager.cc:5495] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec reported cstate change: term changed from 0 to 1, leader changed from <none> to bb343de1be3245a6867b8f1a4f0d60ec (127.30.146.130). New cstate: current_term: 1 leader_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb343de1be3245a6867b8f1a4f0d60ec" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 41101 } health_report { overall_health: HEALTHY } } }
I20260808 10:38:44.549659 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.546328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39502 (local address 127.30.146.130:41101)
0808 10:38:44.546469 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:44.546473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:44.546487 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:38:44.546564 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:44.546566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:44.546621 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:44.546722 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:38:44.546944 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.547440 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.548247 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.548450 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.548526 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.548790 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:38:44.549313 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:38:44.549409 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:44.549475 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 10:38:44.549525 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:38:44.555596 20109 raft_consensus.cc:1261] T 32c23364ae5e498f8e377e95466afcb5 P a96a00aeabba45f48615ed11571bc4c3 [term 1 FOLLOWER]: Refusing update from remote peer bb343de1be3245a6867b8f1a4f0d60ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:44.555595 20398 raft_consensus.cc:1261] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c [term 1 FOLLOWER]: Refusing update from remote peer bb343de1be3245a6867b8f1a4f0d60ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:44.556329 20476 consensus_queue.cc:1035] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96a00aeabba45f48615ed11571bc4c3" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 34927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:44.556515 20472 consensus_queue.cc:1035] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb21091e2bf04cf281142cdcbd97f59c" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:44.563755 20480 mvcc.cc:205] Tried to move back new op lower bound from 7316215908572078080 to 7316215908419375104. Current Snapshot: MvccSnapshot[applied={T|T < 7316215908572078080}]
I20260808 10:38:44.564503 20481 mvcc.cc:205] Tried to move back new op lower bound from 7316215908572078080 to 7316215908419375104. Current Snapshot: MvccSnapshot[applied={T|T < 7316215908572078080}]
I20260808 10:38:44.574455 19950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:44.574646 19950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:44.577231 19950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fed7f16c8c2c48678e5924b096d122c3: Sending DeleteTablet for 3 replicas of tablet 32c23364ae5e498f8e377e95466afcb5
I20260808 10:38:44.578105 20088 tablet_service.cc:1514] Processing DeleteTablet for tablet 32c23364ae5e498f8e377e95466afcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:44 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:45580
I20260808 10:38:44.578361 20234 tablet_service.cc:1514] Processing DeleteTablet for tablet 32c23364ae5e498f8e377e95466afcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:44 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:39496
I20260808 10:38:44.578811 20377 tablet_service.cc:1514] Processing DeleteTablet for tablet 32c23364ae5e498f8e377e95466afcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:44 UTC) from {username='kudu', principal='kudu/127.30.146.190@KRBTEST.COM'} at 127.0.0.1:40382
I20260808 10:38:44.578919 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 20018
I20260808 10:38:44.579780 20491 tablet_replica.cc:330] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: stopping tablet replica
I20260808 10:38:44.580093 20491 raft_consensus.cc:2229] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:38:44.580466 20491 raft_consensus.cc:2258] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:44.582116 20491 ts_tablet_manager.cc:1905] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:44.585256 20492 tablet_replica.cc:330] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: stopping tablet replica
I20260808 10:38:44.585539 20491 ts_tablet_manager.cc:1918] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:44.585628 20492 raft_consensus.cc:2229] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:38:44.585667 20491 log.cc:1196] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/32c23364ae5e498f8e377e95466afcb5
I20260808 10:38:44.585996 20492 raft_consensus.cc:2258] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:44.586025 20491 ts_tablet_manager.cc:1939] T 32c23364ae5e498f8e377e95466afcb5 P bb343de1be3245a6867b8f1a4f0d60ec: Deleting consensus metadata
I20260808 10:38:44.587119 19937 catalog_manager.cc:4841] TS bb343de1be3245a6867b8f1a4f0d60ec (127.30.146.130:41101): tablet 32c23364ae5e498f8e377e95466afcb5 (table test-table [id=d159aa48f991494ba0732d5642766fe7]) successfully deleted
I20260808 10:38:44.587601 20492 ts_tablet_manager.cc:1905] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:44.591421 20492 ts_tablet_manager.cc:1918] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:44.591547 20492 log.cc:1196] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/32c23364ae5e498f8e377e95466afcb5
I20260808 10:38:44.591971 20492 ts_tablet_manager.cc:1939] T 32c23364ae5e498f8e377e95466afcb5 P fb21091e2bf04cf281142cdcbd97f59c: Deleting consensus metadata
I20260808 10:38:44.593393 19937 catalog_manager.cc:4841] TS fb21091e2bf04cf281142cdcbd97f59c (127.30.146.131:45067): tablet 32c23364ae5e498f8e377e95466afcb5 (table test-table [id=d159aa48f991494ba0732d5642766fe7]) successfully deleted
W20260808 10:38:44.594331 19938 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:34927 (error 108)
I20260808 10:38:44.595223 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 20164
I20260808 10:38:44.596505 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.595056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:34927 (local address 127.0.0.1:45584)
0808 10:38:44.596320 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:44.596407 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1164}
W20260808 10:38:44.596692 19938 catalog_manager.cc:4584] TS a96a00aeabba45f48615ed11571bc4c3 (127.30.146.129:34927): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 32c23364ae5e498f8e377e95466afcb5: Network error: Client connection negotiation failed: client connection to 127.30.146.129:34927: connect: Connection refused (error 111)
I20260808 10:38:44.605592 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 20309
I20260808 10:38:44.614287 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 19918
2026-08-08T10:38:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2475 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T10:38:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:44Z Disabled control of system clock
I20260808 10:38:44.649533 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33739
--webserver_interface=127.30.146.190
--webserver_port=0
--builtin_ntp_servers=127.30.146.148:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:33739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:38:44.763898 20497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:44.764178 20497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:44.764214 20497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:44.768448 20497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:44.768518 20497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:44.768534 20497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:44.768551 20497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:44.768566 20497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:44.773348 20497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:33739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:44.774394 20497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:44.775827 20497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:44.785449 20502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:44.785478 20503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:44.785481 20505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:44.785537 20497 server_base.cc:1034] running on GCE node
I20260808 10:38:44.786298 20497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:44.787503 20497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:44.788698 20497 hybrid_clock.cc:648] HybridClock initialized: now 1786185524788666 us; error 47 us; skew 500 ppm
I20260808 10:38:44.790601 20497 webserver.cc:460] Webserver started at http://127.30.146.190:40495/ using document root <none> and password file <none>
I20260808 10:38:44.791205 20497 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:44.791306 20497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:44.791539 20497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:44.793682 20497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "c51c891d2a1441a0ba88020fbc779536"
format_stamp: "Formatted at 2026-08-08 10:38:44 on dist-test-slave-9gcw"
server_key: "e4b114972de774438ade842d123ee0ae"
server_key_iv: "05e73d676242a653173ab0e2272b9c60"
server_key_version: "encryptionkey@0"
I20260808 10:38:44.794281 20497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "c51c891d2a1441a0ba88020fbc779536"
format_stamp: "Formatted at 2026-08-08 10:38:44 on dist-test-slave-9gcw"
server_key: "e4b114972de774438ade842d123ee0ae"
server_key_iv: "05e73d676242a653173ab0e2272b9c60"
server_key_version: "encryptionkey@0"
I20260808 10:38:44.798082 20497 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:38:44.800531 20511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:44.801800 20497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:38:44.801959 20497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "c51c891d2a1441a0ba88020fbc779536"
format_stamp: "Formatted at 2026-08-08 10:38:44 on dist-test-slave-9gcw"
server_key: "e4b114972de774438ade842d123ee0ae"
server_key_iv: "05e73d676242a653173ab0e2272b9c60"
server_key_version: "encryptionkey@0"
I20260808 10:38:44.802078 20497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:44.821306 20497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:44.822072 20497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:44.822286 20497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:44.829631 20497 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:33739
I20260808 10:38:44.829653 20567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:33739 every 8 connection(s)
I20260808 10:38:44.830600 20497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:44.833832 20569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:44.835942 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 20497
I20260808 10:38:44.836084 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:44.836433 31306 external_mini_cluster.cc:1428] Setting key ce9b3ebd07cd5e69a0f4ae073814ca84
I20260808 10:38:44.840233 20569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536: Bootstrap starting.
I20260808 10:38:44.842458 20569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:44.843196 20569 log.cc:824] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:44.845178 20569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536: No bootstrap required, opened a new log
I20260808 10:38:44.845974 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:44.837904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59976 (local address 127.30.146.190:33739)
0808 10:38:44.838571 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:44.838586 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:44.838622 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:38:44.839394 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:44.839413 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:44.839800 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:44.840139 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:44.840579 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.841597 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.842492 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:44.843030 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:44.843206 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:44.844118 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:44.844124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:44.844136 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:44.844519 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:44.844575 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:44.844595 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:44.844711 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:44.844848 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 10:38:44.845100 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":134,"threads_started":1}
I20260808 10:38:44.849360 20569 raft_consensus.cc:348] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c891d2a1441a0ba88020fbc779536" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33739 } }
I20260808 10:38:44.849606 20569 raft_consensus.cc:374] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:44.849684 20569 raft_consensus.cc:729] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c51c891d2a1441a0ba88020fbc779536, State: Initialized, Role: FOLLOWER
I20260808 10:38:44.850109 20569 consensus_queue.cc:260] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c891d2a1441a0ba88020fbc779536" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33739 } }
I20260808 10:38:44.850261 20569 raft_consensus.cc:388] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:44.850373 20569 raft_consensus.cc:482] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:44.850546 20569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:44.851889 20569 raft_consensus.cc:504] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c891d2a1441a0ba88020fbc779536" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33739 } }
I20260808 10:38:44.852241 20569 leader_election.cc:302] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c51c891d2a1441a0ba88020fbc779536; no voters:
I20260808 10:38:44.852653 20569 leader_election.cc:288] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:44.852809 20574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:44.853027 20574 raft_consensus.cc:686] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [term 1 LEADER]: Becoming Leader. State: Replica: c51c891d2a1441a0ba88020fbc779536, State: Running, Role: LEADER
I20260808 10:38:44.853367 20574 consensus_queue.cc:237] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c891d2a1441a0ba88020fbc779536" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33739 } }
I20260808 10:38:44.853989 20569 sys_catalog.cc:564] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:44.855187 20576 sys_catalog.cc:455] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c51c891d2a1441a0ba88020fbc779536. Latest consensus state: current_term: 1 leader_uuid: "c51c891d2a1441a0ba88020fbc779536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c891d2a1441a0ba88020fbc779536" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33739 } } }
I20260808 10:38:44.855340 20576 sys_catalog.cc:458] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:44.855633 20575 sys_catalog.cc:455] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c51c891d2a1441a0ba88020fbc779536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51c891d2a1441a0ba88020fbc779536" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33739 } } }
I20260808 10:38:44.855729 20575 sys_catalog.cc:458] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:44.855820 20583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:44.859140 20583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:44.864379 20583 catalog_manager.cc:1347] Generated new cluster ID: 52f4dcdb0845463cafe8a4d3d57a6c15
I20260808 10:38:44.864454 20583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:44.895617 20583 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:44.896916 20583 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:44.909588 20583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536: Generated new TSK 0
I20260808 10:38:44.910552 20583 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:44.913626 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.129:0
--local_ip_for_outbound_sockets=127.30.146.129
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33739
--builtin_ntp_servers=127.30.146.148:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:38:45.036370 20593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:45.036731 20593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:45.036813 20593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:45.041821 20593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:45.041908 20593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:45.042007 20593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.129
I20260808 10:38:45.046988 20593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.146.129
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.129
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:45.048231 20593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:45.049746 20593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:45.060201 20601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:45.060293 20598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:45.060441 20593 server_base.cc:1034] running on GCE node
W20260808 10:38:45.060300 20599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:45.061043 20593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:45.061771 20593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:45.062958 20593 hybrid_clock.cc:648] HybridClock initialized: now 1786185525062928 us; error 54 us; skew 500 ppm
I20260808 10:38:45.064973 20593 webserver.cc:460] Webserver started at http://127.30.146.129:33489/ using document root <none> and password file <none>
I20260808 10:38:45.065641 20593 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:45.065749 20593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:45.065999 20593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:45.068110 20593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/data/instance:
uuid: "290ca3ab8770423b8ac74059a092dcf6"
format_stamp: "Formatted at 2026-08-08 10:38:45 on dist-test-slave-9gcw"
server_key: "de5fde964c63abf9cdebea0c0290cddd"
server_key_iv: "0c70beab026a7940e07b83a0893f9d7b"
server_key_version: "encryptionkey@0"
I20260808 10:38:45.068707 20593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance:
uuid: "290ca3ab8770423b8ac74059a092dcf6"
format_stamp: "Formatted at 2026-08-08 10:38:45 on dist-test-slave-9gcw"
server_key: "de5fde964c63abf9cdebea0c0290cddd"
server_key_iv: "0c70beab026a7940e07b83a0893f9d7b"
server_key_version: "encryptionkey@0"
I20260808 10:38:45.072593 20593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:38:45.075229 20607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:45.076485 20593 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:38:45.076622 20593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/wal
uuid: "290ca3ab8770423b8ac74059a092dcf6"
format_stamp: "Formatted at 2026-08-08 10:38:45 on dist-test-slave-9gcw"
server_key: "de5fde964c63abf9cdebea0c0290cddd"
server_key_iv: "0c70beab026a7940e07b83a0893f9d7b"
server_key_version: "encryptionkey@0"
I20260808 10:38:45.076773 20593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:45.108125 20593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:45.108922 20593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:45.109184 20593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:45.109853 20593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:45.110863 20593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:45.110937 20593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:45.111013 20593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:45.111065 20593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:45.120371 20593 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.129:37557
I20260808 10:38:45.120404 20732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.129:37557 every 8 connection(s)
I20260808 10:38:45.121368 20593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/data/info.pb
I20260808 10:38:45.121743 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 20593
I20260808 10:38:45.121901 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-0/wal/instance
I20260808 10:38:45.122300 31306 external_mini_cluster.cc:1428] Setting key f475f4bc664981d3e7c1c02628bae7f7
I20260808 10:38:45.124650 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.130:0
--local_ip_for_outbound_sockets=127.30.146.130
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33739
--builtin_ntp_servers=127.30.146.148:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:38:45.131927 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.123332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:60211 (local address 127.30.146.190:33739)
0808 10:38:45.123463 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.123467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.124881 (+ 1414us) server_negotiation.cc:407] Connection header received
0808 10:38:45.127048 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.127054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.127121 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.127223 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:45.128105 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.128674 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.129938 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.130164 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.130278 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.131413 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:45.131417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:45.131420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:45.131442 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:45.131461 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:45.131468 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:45.131548 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.131710 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.131775 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:38:45.132815 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.123892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33739 (local address 127.30.146.129:60211)
0808 10:38:45.124698 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.124760 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:45.126827 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:45.127383 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:45.127390 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:45.127475 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:45.127973 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.127985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.128835 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.128839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.129804 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.129814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.130325 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.130330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.130499 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.131135 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:38:45.131137 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:45.131179 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:38:45.131253 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:45.131550 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:45.131563 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:45.131569 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:38:45.131701 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 10:38:45.131990 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":111,"threads_started":1}
I20260808 10:38:45.134150 20733 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33739
I20260808 10:38:45.134457 20733 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:45.135176 20733 heartbeater.cc:511] Master 127.30.146.190:33739 requested a full tablet report, sending...
I20260808 10:38:45.137100 20528 ts_manager.cc:194] Registered new tserver with Master: 290ca3ab8770423b8ac74059a092dcf6 (127.30.146.129:37557)
I20260808 10:38:45.138731 20528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.129:60211
W20260808 10:38:45.242448 20736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:45.242766 20736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:45.242815 20736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:45.246838 20736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:45.246898 20736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:45.246953 20736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.130
I20260808 10:38:45.251636 20736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.146.130
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.130
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:45.252815 20736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:45.254341 20736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:45.264519 20744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:45.264578 20741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:45.264578 20742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:45.264719 20736 server_base.cc:1034] running on GCE node
I20260808 10:38:45.265203 20736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:45.265846 20736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:45.267022 20736 hybrid_clock.cc:648] HybridClock initialized: now 1786185525266998 us; error 34 us; skew 500 ppm
I20260808 10:38:45.268966 20736 webserver.cc:460] Webserver started at http://127.30.146.130:36081/ using document root <none> and password file <none>
I20260808 10:38:45.269615 20736 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:45.269681 20736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:45.269949 20736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:45.272048 20736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/data/instance:
uuid: "f80a1a150e8141b081f1132c974beb50"
format_stamp: "Formatted at 2026-08-08 10:38:45 on dist-test-slave-9gcw"
server_key: "172889e39e328d60727d2ec602b3cee4"
server_key_iv: "06166e7012d26816b3eb5e50a4f53303"
server_key_version: "encryptionkey@0"
I20260808 10:38:45.272634 20736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance:
uuid: "f80a1a150e8141b081f1132c974beb50"
format_stamp: "Formatted at 2026-08-08 10:38:45 on dist-test-slave-9gcw"
server_key: "172889e39e328d60727d2ec602b3cee4"
server_key_iv: "06166e7012d26816b3eb5e50a4f53303"
server_key_version: "encryptionkey@0"
I20260808 10:38:45.276574 20736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:38:45.279126 20751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:45.280452 20736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:38:45.280596 20736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/wal
uuid: "f80a1a150e8141b081f1132c974beb50"
format_stamp: "Formatted at 2026-08-08 10:38:45 on dist-test-slave-9gcw"
server_key: "172889e39e328d60727d2ec602b3cee4"
server_key_iv: "06166e7012d26816b3eb5e50a4f53303"
server_key_version: "encryptionkey@0"
I20260808 10:38:45.280715 20736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:45.297670 20736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:45.298436 20736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:45.298671 20736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:45.299319 20736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:45.300343 20736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:45.300421 20736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:45.300500 20736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:45.300547 20736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:45.311019 20736 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.130:40749
I20260808 10:38:45.311054 20871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.130:40749 every 8 connection(s)
I20260808 10:38:45.312106 20736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/data/info.pb
I20260808 10:38:45.312826 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 20736
I20260808 10:38:45.312971 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/wal/instance
I20260808 10:38:45.313333 31306 external_mini_cluster.cc:1428] Setting key 3d02a3c9b418a74a585704ec2899e4ce
I20260808 10:38:45.315613 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.146.131:0
--local_ip_for_outbound_sockets=127.30.146.131
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33739
--builtin_ntp_servers=127.30.146.148:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:38:45.323666 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.314308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:39165 (local address 127.30.146.190:33739)
0808 10:38:45.314448 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.314452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.315817 (+ 1365us) server_negotiation.cc:407] Connection header received
0808 10:38:45.317319 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.317324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.317393 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.317495 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:45.318769 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.319320 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.321300 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.321519 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.321605 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.323098 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:45.323102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:45.323107 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:45.323126 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:45.323158 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:45.323167 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:45.323272 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.323486 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.323544 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:38:45.325136 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.314857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33739 (local address 127.30.146.130:39165)
0808 10:38:45.315669 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.315722 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:45.317049 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:45.317710 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:45.317721 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:45.317839 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:45.318598 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.318622 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.319473 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.319480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.321107 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.321122 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.321704 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.321711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.322002 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.322731 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:38:45.322736 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:45.322799 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:38:45.322902 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:45.323296 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:45.323318 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:45.323327 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:38:45.323486 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 10:38:45.323844 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":181,"threads_started":1}
I20260808 10:38:45.326488 20873 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33739
I20260808 10:38:45.326757 20873 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:45.327339 20873 heartbeater.cc:511] Master 127.30.146.190:33739 requested a full tablet report, sending...
I20260808 10:38:45.328508 20528 ts_manager.cc:194] Registered new tserver with Master: f80a1a150e8141b081f1132c974beb50 (127.30.146.130:40749)
I20260808 10:38:45.329339 20528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.130:39165
W20260808 10:38:45.436872 20876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:45.437212 20876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:45.437270 20876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:45.441498 20876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:45.441555 20876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:45.441608 20876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.146.131
I20260808 10:38:45.446442 20876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.146.131
--webserver_port=0
--tserver_master_addrs=127.30.146.190:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.146.131
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:45.447515 20876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:45.448896 20876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:45.459416 20882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:45.459511 20876 server_base.cc:1034] running on GCE node
W20260808 10:38:45.459452 20884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:45.459416 20881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:45.460162 20876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:45.460851 20876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:45.462080 20876 hybrid_clock.cc:648] HybridClock initialized: now 1786185525462017 us; error 73 us; skew 500 ppm
I20260808 10:38:45.464445 20876 webserver.cc:460] Webserver started at http://127.30.146.131:33129/ using document root <none> and password file <none>
I20260808 10:38:45.465196 20876 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:45.465323 20876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:45.465629 20876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:45.468001 20876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/data/instance:
uuid: "517cc235c4574cfe877af31bd6a2d72a"
format_stamp: "Formatted at 2026-08-08 10:38:45 on dist-test-slave-9gcw"
server_key: "c52ead1b43cf31c7e87c87f42d0fd6fe"
server_key_iv: "a0e610097a6f9d059509ff37dd57cf1a"
server_key_version: "encryptionkey@0"
I20260808 10:38:45.468720 20876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance:
uuid: "517cc235c4574cfe877af31bd6a2d72a"
format_stamp: "Formatted at 2026-08-08 10:38:45 on dist-test-slave-9gcw"
server_key: "c52ead1b43cf31c7e87c87f42d0fd6fe"
server_key_iv: "a0e610097a6f9d059509ff37dd57cf1a"
server_key_version: "encryptionkey@0"
I20260808 10:38:45.473310 20876 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 10:38:45.476292 20890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:45.477736 20876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:38:45.477906 20876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/wal
uuid: "517cc235c4574cfe877af31bd6a2d72a"
format_stamp: "Formatted at 2026-08-08 10:38:45 on dist-test-slave-9gcw"
server_key: "c52ead1b43cf31c7e87c87f42d0fd6fe"
server_key_iv: "a0e610097a6f9d059509ff37dd57cf1a"
server_key_version: "encryptionkey@0"
I20260808 10:38:45.478034 20876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:45.492710 20876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:45.493563 20876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:45.493777 20876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:45.494405 20876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:38:45.495404 20876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:38:45.495478 20876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:45.495553 20876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:38:45.495595 20876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:45.505064 20876 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.131:38999
I20260808 10:38:45.505235 21009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.131:38999 every 8 connection(s)
I20260808 10:38:45.506174 20876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/data/info.pb
I20260808 10:38:45.513849 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 20876
I20260808 10:38:45.513990 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/wal/instance
I20260808 10:38:45.514376 31306 external_mini_cluster.cc:1428] Setting key ef04873169e51bedc256adde0725fcd4
I20260808 10:38:45.516264 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.508245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.131:57617 (local address 127.30.146.190:33739)
0808 10:38:45.508400 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.508403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.509618 (+ 1215us) server_negotiation.cc:407] Connection header received
0808 10:38:45.511013 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.511016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.511063 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.511141 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:45.512072 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.512625 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.513985 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.514298 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.514385 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.515718 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:45.515722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:45.515725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:45.515739 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:45.515763 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:45.515776 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:45.515853 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.516061 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.516121 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:38:45.517334 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.508700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:33739 (local address 127.30.146.131:57617)
0808 10:38:45.509452 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.509495 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:45.510774 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:45.511307 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:45.511315 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:45.511400 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:38:45.511930 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.511943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.512740 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.512742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.513801 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.513823 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.514471 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.514476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.514682 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.515392 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:38:45.515394 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:38:45.515458 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:38:45.515541 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:38:45.515881 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:38:45.515891 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:38:45.515898 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:38:45.516045 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 10:38:45.516284 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":128,"threads_started":1}
I20260808 10:38:45.518508 21010 heartbeater.cc:347] Connected to a master server at 127.30.146.190:33739
I20260808 10:38:45.518729 21010 heartbeater.cc:464] Registering TS with master...
I20260808 10:38:45.519209 21010 heartbeater.cc:511] Master 127.30.146.190:33739 requested a full tablet report, sending...
I20260808 10:38:45.520161 20528 ts_manager.cc:194] Registered new tserver with Master: 517cc235c4574cfe877af31bd6a2d72a (127.30.146.131:38999)
I20260808 10:38:45.520689 20528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.146.131:57617
I20260808 10:38:45.528522 31306 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:38:45.534448 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.530567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59986 (local address 127.30.146.190:33739)
0808 10:38:45.530692 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.530696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.530913 (+ 217us) server_negotiation.cc:407] Connection header received
0808 10:38:45.530997 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.530999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.531038 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.531104 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:45.531363 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.531870 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.532667 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.532874 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.532948 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.534101 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:45.534104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:45.534106 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:45.534117 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:45.534134 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:45.534141 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:45.534201 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.534291 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.534340 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:38:45.537293 20528 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:38:45.540526 20528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:38:45.555764 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.550074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43360 (local address 127.30.146.131:38999)
0808 10:38:45.551115 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.551128 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.551166 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:38:45.551242 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.551246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.551409 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.551548 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:45.551772 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.552876 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.554121 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.555042 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.555184 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.555401 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.555508 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.555587 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":919,"thread_start_us":170,"threads_started":1}
I20260808 10:38:45.555902 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.549878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:38999 (local address 127.0.0.1:43360)
0808 10:38:45.551007 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.551020 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:45.551096 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:45.551554 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:45.551559 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:45.551578 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:45.551655 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.551660 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.552973 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.552979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.553997 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.554006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.555163 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.555166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.555280 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.555339 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 10:38:45.555461 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 10:38:45.555689 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"mutex_wait_us":94,"thread_start_us":110,"threads_started":1}
I20260808 10:38:45.556177 21022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.549117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:40749 (local address 127.0.0.1:39000)
0808 10:38:45.549691 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.549744 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:45.549877 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:45.550809 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:45.550817 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:45.550845 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:45.551002 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.551010 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.552630 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.552633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.554245 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.554251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.555276 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.555279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.555371 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.555388 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:38:45.555444 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:38:45.555683 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":338,"thread_start_us":110,"threads_started":1}
I20260808 10:38:45.556442 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.549316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39000 (local address 127.30.146.130:40749)
0808 10:38:45.550193 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.550209 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.550234 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:38:45.550318 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.550325 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.550564 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.550736 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:45.551142 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.552371 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.554377 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.555169 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.555280 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.555426 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.555483 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.555578 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":124,"threads_started":1}
I20260808 10:38:45.557860 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.549326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:37557 (local address 127.0.0.1:40830)
0808 10:38:45.550754 (+ 1428us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.550771 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:45.550861 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:45.551304 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:45.551307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:45.551325 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:45.551417 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.551424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.552525 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.552528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.554239 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.554251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.557475 (+ 3224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.557478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.557575 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.557599 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:38:45.557664 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:38:45.557736 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1334,"mutex_wait_us":76,"thread_start_us":122,"threads_started":1}
I20260808 10:38:45.558003 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.550351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.30.146.129:37557)
0808 10:38:45.550746 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.550759 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.550873 (+ 114us) server_negotiation.cc:407] Connection header received
0808 10:38:45.550968 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.550973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.551161 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.551330 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:45.551517 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.552381 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.556738 (+ 4357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.557352 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.557491 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.557677 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.557726 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.557816 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":135,"threads_started":1}
I20260808 10:38:45.559247 20940 tablet_service.cc:1467] Processing CreateTablet for tablet 70f1e3f39b6245c5b4b90c19c561ff95 (DEFAULT_TABLE table=test-table [id=bbb890a28347474aae1c044a6cfd77c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:45.559554 20801 tablet_service.cc:1467] Processing CreateTablet for tablet 70f1e3f39b6245c5b4b90c19c561ff95 (DEFAULT_TABLE table=test-table [id=bbb890a28347474aae1c044a6cfd77c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:45.560467 20801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70f1e3f39b6245c5b4b90c19c561ff95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:45.560467 20940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70f1e3f39b6245c5b4b90c19c561ff95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:45.560537 20660 tablet_service.cc:1467] Processing CreateTablet for tablet 70f1e3f39b6245c5b4b90c19c561ff95 (DEFAULT_TABLE table=test-table [id=bbb890a28347474aae1c044a6cfd77c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:38:45.561442 20660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70f1e3f39b6245c5b4b90c19c561ff95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:45.567358 21027 tablet_bootstrap.cc:492] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6: Bootstrap starting.
I20260808 10:38:45.568634 21029 tablet_bootstrap.cc:492] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: Bootstrap starting.
I20260808 10:38:45.569514 21030 tablet_bootstrap.cc:492] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: Bootstrap starting.
I20260808 10:38:45.569674 21027 tablet_bootstrap.cc:654] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:45.570640 21027 log.cc:824] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:45.570791 21029 tablet_bootstrap.cc:654] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:45.571542 21030 tablet_bootstrap.cc:654] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:45.571620 21029 log.cc:824] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:45.572299 21030 log.cc:824] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:45.572770 21027 tablet_bootstrap.cc:492] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6: No bootstrap required, opened a new log
I20260808 10:38:45.572981 21027 ts_tablet_manager.cc:1397] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 10:38:45.573896 21029 tablet_bootstrap.cc:492] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: No bootstrap required, opened a new log
I20260808 10:38:45.574107 21029 ts_tablet_manager.cc:1397] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:38:45.574636 21030 tablet_bootstrap.cc:492] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: No bootstrap required, opened a new log
I20260808 10:38:45.574867 21030 ts_tablet_manager.cc:1397] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:38:45.576648 21027 raft_consensus.cc:348] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } }
I20260808 10:38:45.576892 21027 raft_consensus.cc:374] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:45.576946 21027 raft_consensus.cc:729] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 290ca3ab8770423b8ac74059a092dcf6, State: Initialized, Role: FOLLOWER
I20260808 10:38:45.577437 21027 consensus_queue.cc:260] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } }
I20260808 10:38:45.578089 20733 heartbeater.cc:503] Master 127.30.146.190:33739 was elected leader, sending a full tablet report...
I20260808 10:38:45.578408 21027 ts_tablet_manager.cc:1428] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260808 10:38:45.578689 21029 raft_consensus.cc:348] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } }
I20260808 10:38:45.578946 21029 raft_consensus.cc:374] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:45.579015 21029 raft_consensus.cc:729] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 517cc235c4574cfe877af31bd6a2d72a, State: Initialized, Role: FOLLOWER
I20260808 10:38:45.579458 21029 consensus_queue.cc:260] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } }
I20260808 10:38:45.580108 21010 heartbeater.cc:503] Master 127.30.146.190:33739 was elected leader, sending a full tablet report...
I20260808 10:38:45.580031 21030 raft_consensus.cc:348] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } }
I20260808 10:38:45.580287 21030 raft_consensus.cc:374] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:45.580338 21030 raft_consensus.cc:729] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80a1a150e8141b081f1132c974beb50, State: Initialized, Role: FOLLOWER
I20260808 10:38:45.580466 21029 ts_tablet_manager.cc:1428] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:38:45.580845 21030 consensus_queue.cc:260] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } }
I20260808 10:38:45.581863 21030 ts_tablet_manager.cc:1428] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260808 10:38:45.582228 20873 heartbeater.cc:503] Master 127.30.146.190:33739 was elected leader, sending a full tablet report...
W20260808 10:38:45.622964 20734 tablet.cc:2369] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:38:45.635392 21036 raft_consensus.cc:482] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:45.635673 21036 raft_consensus.cc:504] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } }
I20260808 10:38:45.636813 21036 leader_election.cc:288] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 290ca3ab8770423b8ac74059a092dcf6 (127.30.146.129:37557), 517cc235c4574cfe877af31bd6a2d72a (127.30.146.131:38999)
I20260808 10:38:45.637977 21034 raft_consensus.cc:482] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:38:45.638180 21034 raft_consensus.cc:504] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } }
I20260808 10:38:45.640254 21034 leader_election.cc:288] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f80a1a150e8141b081f1132c974beb50 (127.30.146.130:40749), 517cc235c4574cfe877af31bd6a2d72a (127.30.146.131:38999)
I20260808 10:38:45.642004 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.637090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:37557 (local address 127.30.146.130:40087)
0808 10:38:45.637288 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.637323 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:45.637433 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:45.637829 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:45.637832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:45.637856 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:45.638011 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.638021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.639030 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.639033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.640285 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.640298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.641533 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.641537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.641647 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.641711 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 10:38:45.641786 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:38:45.641858 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":32}
I20260808 10:38:45.642197 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.637603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:44003 (local address 127.30.146.131:38999)
0808 10:38:45.637749 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.637754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.637957 (+ 203us) server_negotiation.cc:407] Connection header received
0808 10:38:45.638062 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.638067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.638130 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.638257 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:45.638637 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.639258 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.640757 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.641466 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.641559 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.641604 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.641990 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.642067 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:38:45.642611 21037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.637468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:38999 (local address 127.30.146.130:44003)
0808 10:38:45.637842 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.637855 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:45.637965 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:45.638268 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:45.638271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:45.638292 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:45.638503 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.638510 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.639370 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.639375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.640603 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.640611 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.641723 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.641728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.641845 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.641869 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:38:45.642383 (+ 514us) client_negotiation.cc:240] Negotiation successful
0808 10:38:45.642458 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":25,"thread_start_us":126,"threads_started":1}
I20260808 10:38:45.642797 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.637223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.130:40087 (local address 127.30.146.129:37557)
0808 10:38:45.637379 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.637385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.637402 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:38:45.637557 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.637560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.637614 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.637723 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:45.638162 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.638905 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.640528 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.641259 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.641402 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.641467 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.642601 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.642672 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":57}
I20260808 10:38:45.643242 20961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f1e3f39b6245c5b4b90c19c561ff95" candidate_uuid: "f80a1a150e8141b081f1132c974beb50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517cc235c4574cfe877af31bd6a2d72a" is_pre_election: true
I20260808 10:38:45.643263 20682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f1e3f39b6245c5b4b90c19c561ff95" candidate_uuid: "f80a1a150e8141b081f1132c974beb50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "290ca3ab8770423b8ac74059a092dcf6" is_pre_election: true
I20260808 10:38:45.643567 20961 raft_consensus.cc:2454] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f80a1a150e8141b081f1132c974beb50 in term 0.
I20260808 10:38:45.643652 20682 raft_consensus.cc:2454] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f80a1a150e8141b081f1132c974beb50 in term 0.
I20260808 10:38:45.644155 20752 leader_election.cc:302] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 290ca3ab8770423b8ac74059a092dcf6, f80a1a150e8141b081f1132c974beb50; no voters:
I20260808 10:38:45.644466 21036 raft_consensus.cc:2781] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:38:45.644574 21036 raft_consensus.cc:482] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:38:45.644624 21036 raft_consensus.cc:3037] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:45.645469 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.639700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.131:38999 (local address 127.30.146.129:38199)
0808 10:38:45.640523 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.640536 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:45.640618 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:45.641122 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:45.641125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:45.641164 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:45.641295 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.641305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.642403 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.642407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.644068 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.644085 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.645012 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.645016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.645121 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.645169 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 10:38:45.645248 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:38:45.645323 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":4,"thread_start_us":116,"threads_started":1}
I20260808 10:38:45.645465 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.640074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:38199 (local address 127.30.146.131:38999)
0808 10:38:45.640889 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.640898 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.640911 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:38:45.640953 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.640956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.640997 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.641403 (+ 406us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:45.641521 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.642276 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.644192 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.644861 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.645011 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.645058 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.645275 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.645344 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":91,"threads_started":1}
I20260808 10:38:45.645632 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.639934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.146.129:46761 (local address 127.30.146.130:40749)
0808 10:38:45.640053 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.640057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.640289 (+ 232us) server_negotiation.cc:407] Connection header received
0808 10:38:45.640450 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.640453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.640488 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.640529 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:38:45.640966 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.641710 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.644337 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.644933 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.645048 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.645099 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.645455 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.645520 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":56}
I20260808 10:38:45.646092 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.639602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.130:40749 (local address 127.30.146.129:46761)
0808 10:38:45.640212 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.640225 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:38:45.640351 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:38:45.640645 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:38:45.640651 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:38:45.640673 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:38:45.640839 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.640847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.641838 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.641841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.644223 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:38:45.644227 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.645191 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.645195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:38:45.645298 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.645330 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 10:38:45.645814 (+ 484us) client_negotiation.cc:240] Negotiation successful
0808 10:38:45.645886 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":18,"thread_start_us":94,"threads_started":1}
I20260808 10:38:45.646155 21036 raft_consensus.cc:504] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } }
I20260808 10:38:45.646529 21036 leader_election.cc:288] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [CANDIDATE]: Term 1 election: Requested vote from peers 290ca3ab8770423b8ac74059a092dcf6 (127.30.146.129:37557), 517cc235c4574cfe877af31bd6a2d72a (127.30.146.131:38999)
I20260808 10:38:45.646677 20823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f1e3f39b6245c5b4b90c19c561ff95" candidate_uuid: "290ca3ab8770423b8ac74059a092dcf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80a1a150e8141b081f1132c974beb50" is_pre_election: true
I20260808 10:38:45.646922 20961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f1e3f39b6245c5b4b90c19c561ff95" candidate_uuid: "f80a1a150e8141b081f1132c974beb50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517cc235c4574cfe877af31bd6a2d72a"
I20260808 10:38:45.647025 20823 raft_consensus.cc:2379] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 290ca3ab8770423b8ac74059a092dcf6 in current term 1: Already voted for candidate f80a1a150e8141b081f1132c974beb50 in this term.
I20260808 10:38:45.647045 20961 raft_consensus.cc:3037] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:45.647109 20682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f1e3f39b6245c5b4b90c19c561ff95" candidate_uuid: "f80a1a150e8141b081f1132c974beb50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "290ca3ab8770423b8ac74059a092dcf6"
I20260808 10:38:45.647118 20960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70f1e3f39b6245c5b4b90c19c561ff95" candidate_uuid: "290ca3ab8770423b8ac74059a092dcf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517cc235c4574cfe877af31bd6a2d72a" is_pre_election: true
I20260808 10:38:45.647218 20682 raft_consensus.cc:3037] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:45.647778 20609 leader_election.cc:302] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 290ca3ab8770423b8ac74059a092dcf6; no voters: 517cc235c4574cfe877af31bd6a2d72a, f80a1a150e8141b081f1132c974beb50
I20260808 10:38:45.648427 20961 raft_consensus.cc:2454] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f80a1a150e8141b081f1132c974beb50 in term 1.
I20260808 10:38:45.648497 20682 raft_consensus.cc:2454] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f80a1a150e8141b081f1132c974beb50 in term 1.
I20260808 10:38:45.648684 21034 raft_consensus.cc:2726] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 10:38:45.648841 20752 leader_election.cc:302] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 290ca3ab8770423b8ac74059a092dcf6, f80a1a150e8141b081f1132c974beb50; no voters:
I20260808 10:38:45.649026 21036 raft_consensus.cc:2781] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:45.649287 21036 raft_consensus.cc:686] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 1 LEADER]: Becoming Leader. State: Replica: f80a1a150e8141b081f1132c974beb50, State: Running, Role: LEADER
I20260808 10:38:45.649616 21036 consensus_queue.cc:237] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } }
I20260808 10:38:45.652978 20528 catalog_manager.cc:5495] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f80a1a150e8141b081f1132c974beb50 (127.30.146.130). New cstate: current_term: 1 leader_uuid: "f80a1a150e8141b081f1132c974beb50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80a1a150e8141b081f1132c974beb50" member_type: VOTER last_known_addr { host: "127.30.146.130" port: 40749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 } health_report { overall_health: UNKNOWN } } }
I20260808 10:38:45.684671 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.680533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.30.146.130:40749)
0808 10:38:45.680669 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:45.680674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:45.680709 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:38:45.680803 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:45.680805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:45.680869 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:45.680984 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:38:45.681303 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.681847 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.682825 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:45.683074 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:45.683189 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:45.683525 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:38:45.683986 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:38:45.684133 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:45.684175 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 10:38:45.684234 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:38:45.691031 20961 raft_consensus.cc:1261] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a [term 1 FOLLOWER]: Refusing update from remote peer f80a1a150e8141b081f1132c974beb50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:45.691087 20682 raft_consensus.cc:1261] T 70f1e3f39b6245c5b4b90c19c561ff95 P 290ca3ab8770423b8ac74059a092dcf6 [term 1 FOLLOWER]: Refusing update from remote peer f80a1a150e8141b081f1132c974beb50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:38:45.691777 21036 consensus_queue.cc:1035] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "290ca3ab8770423b8ac74059a092dcf6" member_type: VOTER last_known_addr { host: "127.30.146.129" port: 37557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:45.692047 21041 consensus_queue.cc:1035] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "517cc235c4574cfe877af31bd6a2d72a" member_type: VOTER last_known_addr { host: "127.30.146.131" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:38:45.700394 21045 mvcc.cc:205] Tried to move back new op lower bound from 7316215913222152192 to 7316215913063665664. Current Snapshot: MvccSnapshot[applied={T|T < 7316215913222152192}]
I20260808 10:38:45.700733 21047 mvcc.cc:205] Tried to move back new op lower bound from 7316215913222152192 to 7316215913063665664. Current Snapshot: MvccSnapshot[applied={T|T < 7316215913222152192}]
I20260808 10:38:45.709671 20528 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:45.709870 20528 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:38:45.712525 20528 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c51c891d2a1441a0ba88020fbc779536: Sending DeleteTablet for 3 replicas of tablet 70f1e3f39b6245c5b4b90c19c561ff95
I20260808 10:38:45.713706 20660 tablet_service.cc:1514] Processing DeleteTablet for tablet 70f1e3f39b6245c5b4b90c19c561ff95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:45 UTC) from {username='slave'} at 127.0.0.1:40830
I20260808 10:38:45.713709 20940 tablet_service.cc:1514] Processing DeleteTablet for tablet 70f1e3f39b6245c5b4b90c19c561ff95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:45 UTC) from {username='slave'} at 127.0.0.1:43360
I20260808 10:38:45.714025 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 20593
I20260808 10:38:45.714666 20801 tablet_service.cc:1514] Processing DeleteTablet for tablet 70f1e3f39b6245c5b4b90c19c561ff95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:38:45 UTC) from {username='slave'} at 127.0.0.1:39000
I20260808 10:38:45.715525 21058 tablet_replica.cc:330] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: stopping tablet replica
I20260808 10:38:45.715884 21058 raft_consensus.cc:2229] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:38:45.716296 21058 raft_consensus.cc:2258] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:45.718139 21058 ts_tablet_manager.cc:1905] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:45.719354 21056 tablet_replica.cc:330] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: stopping tablet replica
I20260808 10:38:45.719834 21056 raft_consensus.cc:2229] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:38:45.720271 21056 raft_consensus.cc:2258] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:38:45.721769 21058 ts_tablet_manager.cc:1918] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:45.721886 21058 log.cc:1196] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-1/wal/wals/70f1e3f39b6245c5b4b90c19c561ff95
I20260808 10:38:45.722268 21058 ts_tablet_manager.cc:1939] T 70f1e3f39b6245c5b4b90c19c561ff95 P f80a1a150e8141b081f1132c974beb50: Deleting consensus metadata
I20260808 10:38:45.722434 21056 ts_tablet_manager.cc:1905] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:38:45.723774 20513 catalog_manager.cc:4841] TS f80a1a150e8141b081f1132c974beb50 (127.30.146.130:40749): tablet 70f1e3f39b6245c5b4b90c19c561ff95 (table test-table [id=bbb890a28347474aae1c044a6cfd77c8]) successfully deleted
I20260808 10:38:45.726217 21056 ts_tablet_manager.cc:1918] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:38:45.726341 21056 log.cc:1196] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: Deleting WAL directory at /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185256684052-31306-0/minicluster-data/ts-2/wal/wals/70f1e3f39b6245c5b4b90c19c561ff95
I20260808 10:38:45.726706 21056 ts_tablet_manager.cc:1939] T 70f1e3f39b6245c5b4b90c19c561ff95 P 517cc235c4574cfe877af31bd6a2d72a: Deleting consensus metadata
W20260808 10:38:45.727676 20512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.146.129:37557 (error 108)
I20260808 10:38:45.727924 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 20736
I20260808 10:38:45.728207 20512 catalog_manager.cc:4841] TS 517cc235c4574cfe877af31bd6a2d72a (127.30.146.131:38999): tablet 70f1e3f39b6245c5b4b90c19c561ff95 (table test-table [id=bbb890a28347474aae1c044a6cfd77c8]) successfully deleted
I20260808 10:38:45.734397 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:45.734089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.129:37557 (local address 127.0.0.1:40844)
0808 10:38:45.734241 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:38:45.734313 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.129:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 10:38:45.734586 20512 catalog_manager.cc:4584] TS 290ca3ab8770423b8ac74059a092dcf6 (127.30.146.129:37557): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70f1e3f39b6245c5b4b90c19c561ff95: Network error: Client connection negotiation failed: client connection to 127.30.146.129:37557: connect: Connection refused (error 111)
I20260808 10:38:45.739707 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 20876
I20260808 10:38:45.747550 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 20497
2026-08-08T10:38:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1131 ms)
[----------] 6 tests from EncryptionPolicyTest (11093 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:45 dist-test-slave-9gcw krb5kdc[21063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:45 dist-test-slave-9gcw krb5kdc[21063](info): set up 2 sockets
Aug 08 10:38:45 dist-test-slave-9gcw krb5kdc[21063](info): commencing operation
krb5kdc: starting...
W20260808 10:38:46.868175 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:46 dist-test-slave-9gcw krb5kdc[21063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:38:47.013451 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:47.132200 21078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:47.132608 21078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:47.132692 21078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:47.136792 21078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:47.136904 21078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:47.136943 21078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:47.136983 21078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:47.137018 21078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:47.142532 21078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42967
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:47.143748 21078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:47.144835 21078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:47.153672 21086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:47.153793 21078 server_base.cc:1034] running on GCE node
W20260808 10:38:47.153798 21083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:47.153693 21084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:47.154629 21078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:47.155969 21078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:47.157217 21078 hybrid_clock.cc:648] HybridClock initialized: now 1786185527157215 us; error 51 us; skew 500 ppm
Aug 08 10:38:47 dist-test-slave-9gcw krb5kdc[21063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:47.160395 21078 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:38:47.161571 21078 webserver.cc:460] Webserver started at http://127.0.0.1:43193/ using document root <none> and password file <none>
I20260808 10:38:47.162189 21078 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:47.162287 21078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:47.162544 21078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:47.164707 21078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "fd30375bb3844591b09b1a71a70c12fe"
format_stamp: "Formatted at 2026-08-08 10:38:47 on dist-test-slave-9gcw"
server_key: "20e076e5e2537a97e0daaeb6c28ab276"
server_key_iv: "48ac4c6ade35e604539c725bc9c15892"
server_key_version: "encryptionkey@0"
I20260808 10:38:47.165369 21078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "fd30375bb3844591b09b1a71a70c12fe"
format_stamp: "Formatted at 2026-08-08 10:38:47 on dist-test-slave-9gcw"
server_key: "20e076e5e2537a97e0daaeb6c28ab276"
server_key_iv: "48ac4c6ade35e604539c725bc9c15892"
server_key_version: "encryptionkey@0"
I20260808 10:38:47.169346 21078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:38:47.171849 21094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:47.173264 21078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:38:47.173409 21078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "fd30375bb3844591b09b1a71a70c12fe"
format_stamp: "Formatted at 2026-08-08 10:38:47 on dist-test-slave-9gcw"
server_key: "20e076e5e2537a97e0daaeb6c28ab276"
server_key_iv: "48ac4c6ade35e604539c725bc9c15892"
server_key_version: "encryptionkey@0"
I20260808 10:38:47.173532 21078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:47.196343 21078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:47.199950 21078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:47.200184 21078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:47.208114 21078 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42967
I20260808 10:38:47.208168 21150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42967 every 8 connection(s)
I20260808 10:38:47.209249 21078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:47.211638 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 21078
I20260808 10:38:47.211793 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:47.212127 31306 external_mini_cluster.cc:1428] Setting key 0aca5ccfc87950bdcaf0849ce8a0985c
I20260808 10:38:47.212522 21152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:47.220039 21152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe: Bootstrap starting.
Aug 08 10:38:47 dist-test-slave-9gcw krb5kdc[21063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:38:47.222949 21152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:47.223901 21152 log.cc:824] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:47.226667 21152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe: No bootstrap required, opened a new log
I20260808 10:38:47.228062 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:47.213638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53921 (local address 127.0.0.1:42967)
0808 10:38:47.214088 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:47.214105 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:47.214140 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:38:47.214792 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:47.214809 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:47.215162 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:47.215463 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:47.216441 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:47.217501 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:47.218716 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:47.219175 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:47.219289 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:47.222217 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:47.222223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:47.222241 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:47.222273 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:47.224257 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:47.224956 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:47.224967 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:47.224977 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:47.225139 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:47.225537 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:47.225541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:47.225543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:47.226217 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:47.226469 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:47.226731 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 10:38:47.226994 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":159,"threads_started":1}
I20260808 10:38:47.231459 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:38:47.231873 21152 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30375bb3844591b09b1a71a70c12fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } }
I20260808 10:38:47.232127 21152 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:47.232223 21152 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd30375bb3844591b09b1a71a70c12fe, State: Initialized, Role: FOLLOWER
I20260808 10:38:47.232734 21152 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30375bb3844591b09b1a71a70c12fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } }
I20260808 10:38:47.232949 21152 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:47.233044 21152 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:47.233223 21152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:47.234582 21152 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30375bb3844591b09b1a71a70c12fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } }
I20260808 10:38:47.234970 21152 leader_election.cc:302] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd30375bb3844591b09b1a71a70c12fe; no voters:
I20260808 10:38:47.235345 21152 leader_election.cc:288] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:47.235438 21162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:47.235671 21162 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [term 1 LEADER]: Becoming Leader. State: Replica: fd30375bb3844591b09b1a71a70c12fe, State: Running, Role: LEADER
I20260808 10:38:47.236011 21162 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30375bb3844591b09b1a71a70c12fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } }
I20260808 10:38:47.236765 21152 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:47.237459 21164 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd30375bb3844591b09b1a71a70c12fe. Latest consensus state: current_term: 1 leader_uuid: "fd30375bb3844591b09b1a71a70c12fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30375bb3844591b09b1a71a70c12fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } } }
I20260808 10:38:47.237597 21164 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:47.237881 21163 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd30375bb3844591b09b1a71a70c12fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd30375bb3844591b09b1a71a70c12fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } } }
I20260808 10:38:47.237985 21163 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:47.238322 21168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:47.242002 21168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:47.245961 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:47.233918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39533 (local address 127.0.0.1:42967)
0808 10:38:47.234037 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:47.234041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:47.235792 (+ 1751us) server_negotiation.cc:407] Connection header received
0808 10:38:47.235995 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:47.235997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:47.236047 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:47.236099 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:47.237091 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:47.238041 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:47.239192 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:47.239533 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:47.239641 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:47.241766 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:47.241773 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:47.241782 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:47.241809 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:47.243715 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:47.244317 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:47.244321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:47.244324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:47.244425 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:47.244854 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:47.244858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:47.244860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:47.245082 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:47.245321 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:47.245642 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 10:38:47.245754 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:38:47.248912 21168 catalog_manager.cc:1347] Generated new cluster ID: debddab361c3457aa813e203844e34b0
I20260808 10:38:47.248987 21168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:47.256189 21168 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:47.257395 21168 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:47.273741 21168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd30375bb3844591b09b1a71a70c12fe: Generated new TSK 0
I20260808 10:38:47.274695 21168 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:47.281586 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 21078
W20260808 10:38:47.290014 21061 connection.cc:535] client connection to 127.0.0.1:42967 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42967): Connection reset by peer (error 104)
2026-08-08T10:38:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:47 dist-test-slave-9gcw krb5kdc[21184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:47 dist-test-slave-9gcw krb5kdc[21184](info): set up 2 sockets
Aug 08 10:38:47 dist-test-slave-9gcw krb5kdc[21184](info): commencing operation
krb5kdc: starting...
W20260808 10:38:48.441311 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:48 dist-test-slave-9gcw krb5kdc[21184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:38:48.576915 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:48.691870 21197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:48.692144 21197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:48.692219 21197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:48.696442 21197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:48.696506 21197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:48.696527 21197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:48.696542 21197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:48.696587 21197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:48.701853 21197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37443
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:48.702950 21197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:48.704006 21197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:48.712818 21202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:48.712813 21205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:48.712872 21197 server_base.cc:1034] running on GCE node
W20260808 10:38:48.712855 21203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:48.713654 21197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:48.714802 21197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:48.715977 21197 hybrid_clock.cc:648] HybridClock initialized: now 1786185528715957 us; error 30 us; skew 500 ppm
Aug 08 10:38:48 dist-test-slave-9gcw krb5kdc[21184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185528, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:48.718822 21197 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:38:48.719743 21197 webserver.cc:460] Webserver started at http://127.0.0.1:34271/ using document root <none> and password file <none>
I20260808 10:38:48.720340 21197 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:48.720439 21197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:48.720677 21197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:48.722810 21197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "ada18f8bbaff4073b1afa3417d56e39e"
format_stamp: "Formatted at 2026-08-08 10:38:48 on dist-test-slave-9gcw"
server_key: "a0e8c74dec67498f6bcbfaf593f070ae"
server_key_iv: "f6dbdd988e4d7bc4c1763d5c5c85d462"
server_key_version: "encryptionkey@0"
I20260808 10:38:48.723384 21197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "ada18f8bbaff4073b1afa3417d56e39e"
format_stamp: "Formatted at 2026-08-08 10:38:48 on dist-test-slave-9gcw"
server_key: "a0e8c74dec67498f6bcbfaf593f070ae"
server_key_iv: "f6dbdd988e4d7bc4c1763d5c5c85d462"
server_key_version: "encryptionkey@0"
I20260808 10:38:48.727197 21197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:38:48.729668 21212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:48.730937 21197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:38:48.731087 21197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "ada18f8bbaff4073b1afa3417d56e39e"
format_stamp: "Formatted at 2026-08-08 10:38:48 on dist-test-slave-9gcw"
server_key: "a0e8c74dec67498f6bcbfaf593f070ae"
server_key_iv: "f6dbdd988e4d7bc4c1763d5c5c85d462"
server_key_version: "encryptionkey@0"
I20260808 10:38:48.731205 21197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:48.743397 21197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:48.746865 21197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:48.747095 21197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:48.754688 21197 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37443
I20260808 10:38:48.754715 21269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37443 every 8 connection(s)
I20260808 10:38:48.755702 21197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:48.758672 21270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:48.764263 21270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e: Bootstrap starting.
I20260808 10:38:48.764549 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 21197
I20260808 10:38:48.764694 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:48.765018 31306 external_mini_cluster.cc:1428] Setting key 8ac2ed67c64d63a541e1d0dfb9da5a84
I20260808 10:38:48.767103 21270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:48.767863 21270 log.cc:824] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:48.770154 21270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e: No bootstrap required, opened a new log
I20260808 10:38:48.774617 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:48.766483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44817 (local address 127.0.0.1:37443)
0808 10:38:48.766919 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:48.766936 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:48.766973 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:38:48.767672 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:48.767691 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:48.768104 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:48.768493 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:48.768915 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:48.769970 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:48.771163 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:48.771640 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:48.771769 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:48.772733 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:48.772737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:48.772749 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:48.773215 (+ 466us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:48.773275 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:48.773299 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:48.773412 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:48.773638 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 10:38:48.773821 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":147,"threads_started":1}
I20260808 10:38:48.774564 21270 raft_consensus.cc:348] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada18f8bbaff4073b1afa3417d56e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37443 } }
I20260808 10:38:48.774832 21270 raft_consensus.cc:374] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:48.774951 21270 raft_consensus.cc:729] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada18f8bbaff4073b1afa3417d56e39e, State: Initialized, Role: FOLLOWER
I20260808 10:38:48.775400 21270 consensus_queue.cc:260] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada18f8bbaff4073b1afa3417d56e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37443 } }
I20260808 10:38:48.775525 21270 raft_consensus.cc:388] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:48.775620 21270 raft_consensus.cc:482] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:48.775741 21270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:48.776944 21270 raft_consensus.cc:504] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada18f8bbaff4073b1afa3417d56e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37443 } }
I20260808 10:38:48.777335 21270 leader_election.cc:302] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ada18f8bbaff4073b1afa3417d56e39e; no voters:
I20260808 10:38:48.777711 21270 leader_election.cc:288] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:48.777817 21275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:48.777915 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:38:48.778071 21275 raft_consensus.cc:686] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [term 1 LEADER]: Becoming Leader. State: Replica: ada18f8bbaff4073b1afa3417d56e39e, State: Running, Role: LEADER
I20260808 10:38:48.778374 21275 consensus_queue.cc:237] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada18f8bbaff4073b1afa3417d56e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37443 } }
I20260808 10:38:48.778939 21270 sys_catalog.cc:564] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:48.779676 21275 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ada18f8bbaff4073b1afa3417d56e39e. Latest consensus state: current_term: 1 leader_uuid: "ada18f8bbaff4073b1afa3417d56e39e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada18f8bbaff4073b1afa3417d56e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37443 } } }
I20260808 10:38:48.779788 21275 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:48.780153 21276 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ada18f8bbaff4073b1afa3417d56e39e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada18f8bbaff4073b1afa3417d56e39e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37443 } } }
I20260808 10:38:48.780249 21276 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:48.781054 21288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:48.784493 21288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:48.785368 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:48.780257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43807 (local address 127.0.0.1:37443)
0808 10:38:48.780410 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:48.780414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:48.780687 (+ 273us) server_negotiation.cc:407] Connection header received
0808 10:38:48.780917 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:48.780922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:48.780983 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:48.781035 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:48.781860 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:48.782602 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:48.783472 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:48.783720 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:48.783836 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:48.784802 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:48.784805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:48.784808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:48.784821 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:48.784855 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:48.784865 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:48.784963 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:48.785118 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:38:48.785221 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:38:48.790234 21288 catalog_manager.cc:1347] Generated new cluster ID: 50ad411939b045e3b8f5611f562a339e
I20260808 10:38:48.790372 21288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:48.803915 21288 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:48.804991 21288 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:48.815887 21288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ada18f8bbaff4073b1afa3417d56e39e: Generated new TSK 0
I20260808 10:38:48.816869 21288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:48.823956 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 21197
W20260808 10:38:48.833442 21182 connection.cc:535] client connection to 127.0.0.1:37443 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37443): Connection reset by peer (error 104)
2026-08-08T10:38:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:48 dist-test-slave-9gcw krb5kdc[21302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:48 dist-test-slave-9gcw krb5kdc[21302](info): set up 2 sockets
Aug 08 10:38:48 dist-test-slave-9gcw krb5kdc[21302](info): commencing operation
krb5kdc: starting...
W20260808 10:38:49.924448 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:50 dist-test-slave-9gcw krb5kdc[21302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:38:50.062295 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:50.178763 21315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:50.179122 21315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:50.179214 21315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:50.183349 21315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:50.183446 21315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:50.183485 21315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:50.183523 21315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:50.183557 21315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:50.188999 21315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34941
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:50.190181 21315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:50.191252 21315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:50.200055 21322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:50.200085 21324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:50.200239 21315 server_base.cc:1034] running on GCE node
W20260808 10:38:50.200085 21321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:50.200912 21315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:50.202102 21315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:50.203315 21315 hybrid_clock.cc:648] HybridClock initialized: now 1786185530203302 us; error 26 us; skew 500 ppm
Aug 08 10:38:50 dist-test-slave-9gcw krb5kdc[21302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185530, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:50.211918 21315 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:38:50.212977 21315 webserver.cc:460] Webserver started at http://127.0.0.1:38939/ using document root <none> and password file <none>
I20260808 10:38:50.213618 21315 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:50.213698 21315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:50.213884 21315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:50.216136 21315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "1339b40232f345869b954216081b006e"
format_stamp: "Formatted at 2026-08-08 10:38:50 on dist-test-slave-9gcw"
server_key: "3520fb8ee286a9f24dcb325a0442c4f1"
server_key_iv: "009374c5bc088d5ec7660a7a11290401"
server_key_version: "encryptionkey@0"
I20260808 10:38:50.216678 21315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "1339b40232f345869b954216081b006e"
format_stamp: "Formatted at 2026-08-08 10:38:50 on dist-test-slave-9gcw"
server_key: "3520fb8ee286a9f24dcb325a0442c4f1"
server_key_iv: "009374c5bc088d5ec7660a7a11290401"
server_key_version: "encryptionkey@0"
I20260808 10:38:50.220726 21315 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:38:50.223377 21331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:50.224678 21315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:38:50.224817 21315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "1339b40232f345869b954216081b006e"
format_stamp: "Formatted at 2026-08-08 10:38:50 on dist-test-slave-9gcw"
server_key: "3520fb8ee286a9f24dcb325a0442c4f1"
server_key_iv: "009374c5bc088d5ec7660a7a11290401"
server_key_version: "encryptionkey@0"
I20260808 10:38:50.224908 21315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:50.229660 21315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:50.232780 21315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:50.232939 21315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:50.240581 21315 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34941
I20260808 10:38:50.240605 21387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34941 every 8 connection(s)
I20260808 10:38:50.241657 21315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:50.244469 21388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:50.250113 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 21315
I20260808 10:38:50.250275 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:50.250576 31306 external_mini_cluster.cc:1428] Setting key 1f0ad1a4c8ac83d867e118702e68eedb
I20260808 10:38:50.250679 21388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e: Bootstrap starting.
I20260808 10:38:50.253429 21388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:50.254218 21388 log.cc:824] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:50.256137 21391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:50.252023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49779 (local address 127.0.0.1:34941)
0808 10:38:50.252504 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:50.252519 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:50.252555 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:38:50.253303 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:50.253324 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:50.253663 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:50.254036 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:50.254886 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:50.254896 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:50.254908 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:50.254918 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:50.254970 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:50.254989 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:50.255109 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:50.255211 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 10:38:50.255391 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":138,"threads_started":1}
I20260808 10:38:50.256539 21388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e: No bootstrap required, opened a new log
I20260808 10:38:50.259240 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:38:50.260663 21388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1339b40232f345869b954216081b006e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34941 } }
I20260808 10:38:50.260957 21388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:50.261036 21388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1339b40232f345869b954216081b006e, State: Initialized, Role: FOLLOWER
I20260808 10:38:50.261569 21388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1339b40232f345869b954216081b006e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34941 } }
I20260808 10:38:50.261827 21388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:50.261921 21388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:50.262049 21388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:50.263274 21388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1339b40232f345869b954216081b006e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34941 } }
I20260808 10:38:50.263634 21391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:50.261650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49875 (local address 127.0.0.1:34941)
0808 10:38:50.261790 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:50.261796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:50.262039 (+ 243us) server_negotiation.cc:407] Connection header received
0808 10:38:50.262195 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:50.262202 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:50.262250 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:50.262345 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:50.263236 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:50.263239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:50.263242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:50.263262 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:50.263287 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:50.263298 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:50.263392 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:50.263472 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 10:38:50.263513 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:38:50.263703 21388 leader_election.cc:302] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1339b40232f345869b954216081b006e; no voters:
I20260808 10:38:50.264086 21388 leader_election.cc:288] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:50.264178 21399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:50.264402 21399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [term 1 LEADER]: Becoming Leader. State: Replica: 1339b40232f345869b954216081b006e, State: Running, Role: LEADER
I20260808 10:38:50.264798 21399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1339b40232f345869b954216081b006e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34941 } }
I20260808 10:38:50.265610 21388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:50.266496 21401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1339b40232f345869b954216081b006e. Latest consensus state: current_term: 1 leader_uuid: "1339b40232f345869b954216081b006e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1339b40232f345869b954216081b006e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34941 } } }
I20260808 10:38:50.266948 21401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:50.266836 21400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1339b40232f345869b954216081b006e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1339b40232f345869b954216081b006e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34941 } } }
I20260808 10:38:50.267290 21400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:50.267915 21408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:50.271670 21408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:50.277860 21408 catalog_manager.cc:1347] Generated new cluster ID: 00bd644a1663461397e91e1124b8388a
I20260808 10:38:50.277951 21408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:50.291159 21408 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:50.291534 21408 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:50.304658 21408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1339b40232f345869b954216081b006e: Generated new TSK 0
I20260808 10:38:50.305490 21408 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:50.311880 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 21315
2026-08-08T10:38:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:50 dist-test-slave-9gcw krb5kdc[21421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:50 dist-test-slave-9gcw krb5kdc[21421](info): set up 2 sockets
Aug 08 10:38:50 dist-test-slave-9gcw krb5kdc[21421](info): commencing operation
krb5kdc: starting...
W20260808 10:38:51.433933 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:51 dist-test-slave-9gcw krb5kdc[21421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:38:51.577195 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:51.694780 21435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:51.695086 21435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:51.695163 21435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:51.699275 21435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:51.699378 21435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:51.699415 21435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:51.699473 21435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:51.699512 21435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:51.704965 21435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:51.706142 21435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:51.707213 21435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:51.716145 21443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:51.716228 21440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:51.716228 21441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:51.716277 21435 server_base.cc:1034] running on GCE node
I20260808 10:38:51.717043 21435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:51.718331 21435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:51.719542 21435 hybrid_clock.cc:648] HybridClock initialized: now 1786185531719496 us; error 63 us; skew 500 ppm
Aug 08 10:38:51 dist-test-slave-9gcw krb5kdc[21421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:51.722681 21435 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:38:51.723680 21435 webserver.cc:460] Webserver started at http://127.0.0.1:43899/ using document root <none> and password file <none>
I20260808 10:38:51.724313 21435 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:51.724411 21435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:51.724648 21435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:51.726907 21435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "c5adab422eb545a6a0c3a4f4d5e567b8"
format_stamp: "Formatted at 2026-08-08 10:38:51 on dist-test-slave-9gcw"
server_key: "5f221e35ad0c7aa36258021b98dadb35"
server_key_iv: "1611b35f2a14da0f1fc11d0fc9dc66fc"
server_key_version: "encryptionkey@0"
I20260808 10:38:51.727473 21435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "c5adab422eb545a6a0c3a4f4d5e567b8"
format_stamp: "Formatted at 2026-08-08 10:38:51 on dist-test-slave-9gcw"
server_key: "5f221e35ad0c7aa36258021b98dadb35"
server_key_iv: "1611b35f2a14da0f1fc11d0fc9dc66fc"
server_key_version: "encryptionkey@0"
I20260808 10:38:51.731591 21435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:38:51.734200 21450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:51.735570 21435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:38:51.735679 21435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "c5adab422eb545a6a0c3a4f4d5e567b8"
format_stamp: "Formatted at 2026-08-08 10:38:51 on dist-test-slave-9gcw"
server_key: "5f221e35ad0c7aa36258021b98dadb35"
server_key_iv: "1611b35f2a14da0f1fc11d0fc9dc66fc"
server_key_version: "encryptionkey@0"
I20260808 10:38:51.735826 21435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:51.762974 21435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:51.766319 21435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:51.766489 21435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:51.774113 21435 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43359
I20260808 10:38:51.774163 21505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43359 every 8 connection(s)
I20260808 10:38:51.775230 21435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:51.778169 21506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:51.783960 21506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8: Bootstrap starting.
I20260808 10:38:51.785118 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 21435
I20260808 10:38:51.785315 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:51.785607 31306 external_mini_cluster.cc:1428] Setting key 7508341f8726508948722831b2f0f11f
I20260808 10:38:51.787002 21506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:51.787788 21506 log.cc:824] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:51.790125 21506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8: No bootstrap required, opened a new log
Aug 08 10:38:51 dist-test-slave-9gcw krb5kdc[21421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:38:51.794874 21506 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5adab422eb545a6a0c3a4f4d5e567b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43359 } }
I20260808 10:38:51.795176 21506 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:51.795282 21506 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5adab422eb545a6a0c3a4f4d5e567b8, State: Initialized, Role: FOLLOWER
I20260808 10:38:51.795797 21506 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5adab422eb545a6a0c3a4f4d5e567b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43359 } }
I20260808 10:38:51.795976 21506 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:51.796051 21506 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:51.796166 21506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:51.797518 21506 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5adab422eb545a6a0c3a4f4d5e567b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43359 } }
I20260808 10:38:51.797909 21506 leader_election.cc:302] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5adab422eb545a6a0c3a4f4d5e567b8; no voters:
I20260808 10:38:51.798273 21506 leader_election.cc:288] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:51.798405 21511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:51.798635 21511 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [term 1 LEADER]: Becoming Leader. State: Replica: c5adab422eb545a6a0c3a4f4d5e567b8, State: Running, Role: LEADER
I20260808 10:38:51.799067 21511 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5adab422eb545a6a0c3a4f4d5e567b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43359 } }
I20260808 10:38:51.799707 21506 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:51.800856 21512 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5adab422eb545a6a0c3a4f4d5e567b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5adab422eb545a6a0c3a4f4d5e567b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43359 } } }
I20260808 10:38:51.800976 21512 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:51.801363 21513 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5adab422eb545a6a0c3a4f4d5e567b8. Latest consensus state: current_term: 1 leader_uuid: "c5adab422eb545a6a0c3a4f4d5e567b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5adab422eb545a6a0c3a4f4d5e567b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43359 } } }
I20260808 10:38:51.801415 21520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:51.802004 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:51.786764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55291 (local address 127.0.0.1:43359)
0808 10:38:51.787196 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:51.787213 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:51.787249 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:38:51.787929 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:51.787957 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:51.788422 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:51.788765 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:51.789719 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:51.790770 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:51.791932 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:51.792451 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:51.792571 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:51.795769 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:51.795775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:51.795791 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:51.795826 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:51.797902 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:51.798437 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:51.798444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:51.798451 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:51.798614 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:51.798955 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:51.798958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:51.798960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:51.799539 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:51.799751 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:51.800222 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 10:38:51.800782 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":150,"threads_started":1}
I20260808 10:38:51.801493 21513 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:51.805049 21520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:51.805727 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:38:51.811367 21520 catalog_manager.cc:1347] Generated new cluster ID: ba3b4ea6b70b44009e1abf76c56be08b
I20260808 10:38:51.811439 21520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:51.817030 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:51.808187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53743 (local address 127.0.0.1:43359)
0808 10:38:51.808301 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:51.808315 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:51.808600 (+ 285us) server_negotiation.cc:407] Connection header received
0808 10:38:51.808721 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:51.808724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:51.808773 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:51.808865 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:51.809821 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:51.810344 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:51.811160 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:51.811528 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:51.811637 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:51.813442 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:51.813446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:51.813449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:51.813474 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:51.815222 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:51.815753 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:51.815756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:51.815758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:51.815844 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:51.816145 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:51.816148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:51.816149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:51.816353 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:51.816466 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:51.816751 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 10:38:51.816870 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:38:51.828297 21520 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:51.829465 21520 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:51.834563 21520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5adab422eb545a6a0c3a4f4d5e567b8: Generated new TSK 0
I20260808 10:38:51.835603 21520 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:51.842839 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 21435
2026-08-08T10:38:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:51 dist-test-slave-9gcw krb5kdc[21540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:51 dist-test-slave-9gcw krb5kdc[21540](info): set up 2 sockets
Aug 08 10:38:51 dist-test-slave-9gcw krb5kdc[21540](info): commencing operation
krb5kdc: starting...
W20260808 10:38:53.025538 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:53 dist-test-slave-9gcw krb5kdc[21540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:38:53.160382 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:53.278412 21553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:53.278755 21553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:53.278841 21553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:53.282922 21553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:53.283004 21553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:53.283041 21553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:53.283099 21553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:53.283133 21553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:53.288566 21553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:53.289767 21553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:53.290815 21553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:53.300915 21558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:53.301055 21559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:53.301180 21553 server_base.cc:1034] running on GCE node
W20260808 10:38:53.300904 21561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:53.301947 21553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:53.303247 21553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:53.304478 21553 hybrid_clock.cc:648] HybridClock initialized: now 1786185533304466 us; error 27 us; skew 500 ppm
Aug 08 10:38:53 dist-test-slave-9gcw krb5kdc[21540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185533, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:53.307443 21553 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:38:53.308426 21553 webserver.cc:460] Webserver started at http://127.0.0.1:38913/ using document root <none> and password file <none>
I20260808 10:38:53.309005 21553 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:53.309096 21553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:53.309357 21553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:53.311486 21553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "250b3d0d57694a38a4f2ca639f31e47d"
format_stamp: "Formatted at 2026-08-08 10:38:53 on dist-test-slave-9gcw"
server_key: "1df9afab3685bd906e8a62b032413a26"
server_key_iv: "850b0d0a6e6bc71156de39e5510fbc86"
server_key_version: "encryptionkey@0"
I20260808 10:38:53.312067 21553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "250b3d0d57694a38a4f2ca639f31e47d"
format_stamp: "Formatted at 2026-08-08 10:38:53 on dist-test-slave-9gcw"
server_key: "1df9afab3685bd906e8a62b032413a26"
server_key_iv: "850b0d0a6e6bc71156de39e5510fbc86"
server_key_version: "encryptionkey@0"
I20260808 10:38:53.315912 21553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:38:53.318360 21568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:53.319602 21553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:38:53.319787 21553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "250b3d0d57694a38a4f2ca639f31e47d"
format_stamp: "Formatted at 2026-08-08 10:38:53 on dist-test-slave-9gcw"
server_key: "1df9afab3685bd906e8a62b032413a26"
server_key_iv: "850b0d0a6e6bc71156de39e5510fbc86"
server_key_version: "encryptionkey@0"
I20260808 10:38:53.319904 21553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:53.331175 21553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:53.334419 21553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:53.334621 21553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:53.341580 21553 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44773
I20260808 10:38:53.341670 21623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44773 every 8 connection(s)
I20260808 10:38:53.342520 21553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:53.345252 21624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:53.349026 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 21553
I20260808 10:38:53.349220 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:53.349535 31306 external_mini_cluster.cc:1428] Setting key 37d385811caf97ba44a0489a186b100c
I20260808 10:38:53.351279 21624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d: Bootstrap starting.
I20260808 10:38:53.353730 21624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:53.354718 21624 log.cc:824] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:53.357098 21624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d: No bootstrap required, opened a new log
I20260808 10:38:53.358886 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:53.350911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37549 (local address 127.0.0.1:44773)
0808 10:38:53.351311 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:53.351334 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:53.351394 (+ 60us) server_negotiation.cc:407] Connection header received
0808 10:38:53.352048 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:53.352064 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:53.352389 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:53.352660 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:53.352984 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:53.354112 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:53.355360 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:53.355939 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:53.356061 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:53.356979 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:53.356984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:53.356997 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:53.357517 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:53.357575 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:53.357598 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:53.357734 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:53.357857 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 10:38:53.358087 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":142,"threads_started":1}
I20260808 10:38:53.360975 21624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250b3d0d57694a38a4f2ca639f31e47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44773 } }
I20260808 10:38:53.361269 21624 raft_consensus.cc:374] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:53.361353 21624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 250b3d0d57694a38a4f2ca639f31e47d, State: Initialized, Role: FOLLOWER
I20260808 10:38:53.361791 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:38:53.361832 21624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250b3d0d57694a38a4f2ca639f31e47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44773 } }
I20260808 10:38:53.362025 21624 raft_consensus.cc:388] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:53.362092 21624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:53.362190 21624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:53.363184 21624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250b3d0d57694a38a4f2ca639f31e47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44773 } }
I20260808 10:38:53.363557 21624 leader_election.cc:302] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 250b3d0d57694a38a4f2ca639f31e47d; no voters:
I20260808 10:38:53.363893 21624 leader_election.cc:288] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:53.364039 21634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:53.364260 21634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [term 1 LEADER]: Becoming Leader. State: Replica: 250b3d0d57694a38a4f2ca639f31e47d, State: Running, Role: LEADER
I20260808 10:38:53.364565 21634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250b3d0d57694a38a4f2ca639f31e47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44773 } }
I20260808 10:38:53.365191 21624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:53.365916 21636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 250b3d0d57694a38a4f2ca639f31e47d. Latest consensus state: current_term: 1 leader_uuid: "250b3d0d57694a38a4f2ca639f31e47d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250b3d0d57694a38a4f2ca639f31e47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44773 } } }
I20260808 10:38:53.366053 21636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:53.366485 21637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "250b3d0d57694a38a4f2ca639f31e47d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250b3d0d57694a38a4f2ca639f31e47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44773 } } }
I20260808 10:38:53.367460 21637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:53.367762 21643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:53.370874 21643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:53.372169 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:53.364083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34113 (local address 127.0.0.1:44773)
0808 10:38:53.364217 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:53.364224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:53.365476 (+ 1252us) server_negotiation.cc:407] Connection header received
0808 10:38:53.365589 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:53.365592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:53.365643 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:53.365744 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:53.367036 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:53.367841 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:53.370296 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:53.370608 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:53.370726 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:53.371667 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:53.371671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:53.371674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:53.371689 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:53.371720 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:53.371734 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:53.371827 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:53.371964 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 10:38:53.372021 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:38:53.376950 21643 catalog_manager.cc:1347] Generated new cluster ID: eba2a9bef2004b52bb75ac2cf02196b1
I20260808 10:38:53.377017 21643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:53.396355 21643 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:53.397665 21643 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:53.407454 21643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 250b3d0d57694a38a4f2ca639f31e47d: Generated new TSK 0
I20260808 10:38:53.408252 21643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:53.415274 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 21553
2026-08-08T10:38:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1572 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:53 dist-test-slave-9gcw krb5kdc[21656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:53 dist-test-slave-9gcw krb5kdc[21656](info): set up 2 sockets
Aug 08 10:38:53 dist-test-slave-9gcw krb5kdc[21656](info): commencing operation
krb5kdc: starting...
W20260808 10:38:54.537717 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:54 dist-test-slave-9gcw krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:38:54.676349 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41765
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:54.791777 21669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:54.792045 21669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:54.792088 21669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:54.796224 21669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:54.796286 21669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:54.796306 21669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:54.796358 21669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:54.796391 21669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:54.801720 21669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:54.802769 21669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:54.803829 21669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:54.812881 21676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:54.812862 21679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:54.812866 21677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:54.812956 21669 server_base.cc:1034] running on GCE node
I20260808 10:38:54.813750 21669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:54.814913 21669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:54.816110 21669 hybrid_clock.cc:648] HybridClock initialized: now 1786185534816082 us; error 44 us; skew 500 ppm
Aug 08 10:38:54 dist-test-slave-9gcw krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185534, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:54.819025 21669 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:38:54.820052 21669 webserver.cc:460] Webserver started at http://127.0.0.1:35533/ using document root <none> and password file <none>
I20260808 10:38:54.820636 21669 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:54.820737 21669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:54.820981 21669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:54.823396 21669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "d4c1b6f0d2a242429af271d0be6f4262"
format_stamp: "Formatted at 2026-08-08 10:38:54 on dist-test-slave-9gcw"
server_key: "7ca9d0c5152bf11a46c70f32d291786b"
server_key_iv: "5da310dc05e6b77db3fbd66ae31729d2"
server_key_version: "encryptionkey@0"
I20260808 10:38:54.824074 21669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "d4c1b6f0d2a242429af271d0be6f4262"
format_stamp: "Formatted at 2026-08-08 10:38:54 on dist-test-slave-9gcw"
server_key: "7ca9d0c5152bf11a46c70f32d291786b"
server_key_iv: "5da310dc05e6b77db3fbd66ae31729d2"
server_key_version: "encryptionkey@0"
I20260808 10:38:54.828153 21669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:38:54.830783 21686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:54.832017 21669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:38:54.832165 21669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "d4c1b6f0d2a242429af271d0be6f4262"
format_stamp: "Formatted at 2026-08-08 10:38:54 on dist-test-slave-9gcw"
server_key: "7ca9d0c5152bf11a46c70f32d291786b"
server_key_iv: "5da310dc05e6b77db3fbd66ae31729d2"
server_key_version: "encryptionkey@0"
I20260808 10:38:54.832288 21669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:54.837190 21669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:54.840255 21669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:54.840453 21669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:54.848004 21741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41765 every 8 connection(s)
I20260808 10:38:54.848009 21669 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41765
I20260808 10:38:54.848989 21669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:54.851841 21742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:54.853864 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 21669
I20260808 10:38:54.854027 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:54.854336 31306 external_mini_cluster.cc:1428] Setting key 5683faef3f01db306ced2518f8bb5241
I20260808 10:38:54.858104 21742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262: Bootstrap starting.
I20260808 10:38:54.859905 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:54.855802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54573 (local address 127.0.0.1:41765)
0808 10:38:54.856292 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:54.856307 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:54.856345 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:38:54.857113 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:54.857133 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:54.857515 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:54.857854 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:54.858717 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:54.858723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:54.858743 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:54.858754 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:54.858807 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:54.858826 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:54.858976 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:54.859047 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 10:38:54.859209 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":141,"threads_started":1}
I20260808 10:38:54.860641 21742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:54.861624 21742 log.cc:824] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:54.863122 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:38:54.864127 21742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262: No bootstrap required, opened a new log
I20260808 10:38:54.867198 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:54.865525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59973 (local address 127.0.0.1:41765)
0808 10:38:54.865663 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:54.865667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:54.865869 (+ 202us) server_negotiation.cc:407] Connection header received
0808 10:38:54.865988 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:54.865990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:54.866021 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:54.866087 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:54.866794 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:54.866797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:54.866799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:54.866810 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:54.866837 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:54.866847 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:54.866915 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:54.867021 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 10:38:54.867052 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:38:54.868980 21742 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c1b6f0d2a242429af271d0be6f4262" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41765 } }
I20260808 10:38:54.869277 21742 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:54.869336 21742 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c1b6f0d2a242429af271d0be6f4262, State: Initialized, Role: FOLLOWER
I20260808 10:38:54.869802 21742 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c1b6f0d2a242429af271d0be6f4262" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41765 } }
I20260808 10:38:54.869958 21742 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:54.870029 21742 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:54.870136 21742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:54.871106 21742 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c1b6f0d2a242429af271d0be6f4262" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41765 } }
I20260808 10:38:54.871460 21742 leader_election.cc:302] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4c1b6f0d2a242429af271d0be6f4262; no voters:
I20260808 10:38:54.871783 21742 leader_election.cc:288] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:54.871906 21753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:54.872153 21753 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [term 1 LEADER]: Becoming Leader. State: Replica: d4c1b6f0d2a242429af271d0be6f4262, State: Running, Role: LEADER
I20260808 10:38:54.872485 21753 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c1b6f0d2a242429af271d0be6f4262" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41765 } }
I20260808 10:38:54.873116 21742 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:54.874293 21755 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4c1b6f0d2a242429af271d0be6f4262. Latest consensus state: current_term: 1 leader_uuid: "d4c1b6f0d2a242429af271d0be6f4262" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c1b6f0d2a242429af271d0be6f4262" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41765 } } }
I20260808 10:38:54.874426 21755 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:54.874730 21754 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4c1b6f0d2a242429af271d0be6f4262" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c1b6f0d2a242429af271d0be6f4262" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41765 } } }
I20260808 10:38:54.874831 21754 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:54.874966 21764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:54.878257 21764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:54.883834 21764 catalog_manager.cc:1347] Generated new cluster ID: 732d2dce2d8c43a8a221f818560b7520
I20260808 10:38:54.883906 21764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:54.903407 21764 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:54.903785 21764 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:54.908212 21764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4c1b6f0d2a242429af271d0be6f4262: Generated new TSK 0
I20260808 10:38:54.908906 21764 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:54.919468 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 21669
2026-08-08T10:38:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:54 dist-test-slave-9gcw krb5kdc[21777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:54 dist-test-slave-9gcw krb5kdc[21777](info): set up 2 sockets
Aug 08 10:38:54 dist-test-slave-9gcw krb5kdc[21777](info): commencing operation
krb5kdc: starting...
W20260808 10:38:56.120265 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:56 dist-test-slave-9gcw krb5kdc[21777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:38:56.251971 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41599
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:56.387527 21790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:56.387876 21790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:56.387961 21790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:56.392014 21790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:56.392114 21790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:56.392156 21790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:56.392192 21790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:56.392227 21790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:56.397663 21790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:56.398828 21790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:56.399885 21790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:56.408800 21796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:56.408779 21795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:56.408864 21790 server_base.cc:1034] running on GCE node
W20260808 10:38:56.408768 21798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:56.409678 21790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:56.411160 21790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:56.412380 21790 hybrid_clock.cc:648] HybridClock initialized: now 1786185536412290 us; error 107 us; skew 500 ppm
Aug 08 10:38:56 dist-test-slave-9gcw krb5kdc[21777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:56.415233 21790 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:38:56.416255 21790 webserver.cc:460] Webserver started at http://127.0.0.1:33377/ using document root <none> and password file <none>
I20260808 10:38:56.416795 21790 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:56.416859 21790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:56.417027 21790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:56.419183 21790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "3eb065e7ac4e4756a47479ce6d30c241"
format_stamp: "Formatted at 2026-08-08 10:38:56 on dist-test-slave-9gcw"
server_key: "70b00b5e0452c85e393a4cbdd9dd7435"
server_key_iv: "1a8c7651471f4d4b5fe43d1bbeda7ec8"
server_key_version: "encryptionkey@0"
I20260808 10:38:56.419718 21790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "3eb065e7ac4e4756a47479ce6d30c241"
format_stamp: "Formatted at 2026-08-08 10:38:56 on dist-test-slave-9gcw"
server_key: "70b00b5e0452c85e393a4cbdd9dd7435"
server_key_iv: "1a8c7651471f4d4b5fe43d1bbeda7ec8"
server_key_version: "encryptionkey@0"
I20260808 10:38:56.423581 21790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:38:56.426075 21805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:56.427510 21790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:38:56.427659 21790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "3eb065e7ac4e4756a47479ce6d30c241"
format_stamp: "Formatted at 2026-08-08 10:38:56 on dist-test-slave-9gcw"
server_key: "70b00b5e0452c85e393a4cbdd9dd7435"
server_key_iv: "1a8c7651471f4d4b5fe43d1bbeda7ec8"
server_key_version: "encryptionkey@0"
I20260808 10:38:56.427776 21790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:56.447609 21790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:56.450673 21790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:56.450897 21790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:56.458534 21790 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41599
I20260808 10:38:56.458561 21860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41599 every 8 connection(s)
I20260808 10:38:56.459575 21790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:56.462411 21861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:56.468418 21861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241: Bootstrap starting.
I20260808 10:38:56.469748 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 21790
I20260808 10:38:56.469918 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:56.470255 31306 external_mini_cluster.cc:1428] Setting key 5a9a21742e78e27413106697f3f75e1f
I20260808 10:38:56.471645 21861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:56.472445 21861 log.cc:824] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:56.474571 21861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241: No bootstrap required, opened a new log
I20260808 10:38:56.478645 21861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb065e7ac4e4756a47479ce6d30c241" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260808 10:38:56.478876 21861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:56.478963 21861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb065e7ac4e4756a47479ce6d30c241, State: Initialized, Role: FOLLOWER
Aug 08 10:38:56 dist-test-slave-9gcw krb5kdc[21777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:38:56.479543 21861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb065e7ac4e4756a47479ce6d30c241" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260808 10:38:56.479692 21861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:56.479782 21861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:56.479904 21861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:56.481228 21861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb065e7ac4e4756a47479ce6d30c241" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260808 10:38:56.481639 21861 leader_election.cc:302] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eb065e7ac4e4756a47479ce6d30c241; no voters:
I20260808 10:38:56.482021 21861 leader_election.cc:288] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:56.482101 21868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:56.482318 21868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [term 1 LEADER]: Becoming Leader. State: Replica: 3eb065e7ac4e4756a47479ce6d30c241, State: Running, Role: LEADER
I20260808 10:38:56.482666 21868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb065e7ac4e4756a47479ce6d30c241" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260808 10:38:56.483479 21861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:56.483963 21868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3eb065e7ac4e4756a47479ce6d30c241" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb065e7ac4e4756a47479ce6d30c241" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } } }
I20260808 10:38:56.484082 21868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:56.484319 21868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eb065e7ac4e4756a47479ce6d30c241. Latest consensus state: current_term: 1 leader_uuid: "3eb065e7ac4e4756a47479ce6d30c241" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb065e7ac4e4756a47479ce6d30c241" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } } }
I20260808 10:38:56.484429 21868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:56.486738 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:56.471514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:43049 (local address 127.0.0.1:41599)
0808 10:38:56.471994 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:56.472010 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:56.472045 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:38:56.472812 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:56.472839 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:56.473335 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:56.473692 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:56.474519 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:56.475560 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:56.476435 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:56.476920 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:56.477091 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:56.480089 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:56.480094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:56.480108 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:56.480135 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:56.482113 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:56.482752 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:56.482758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:56.482766 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:56.482928 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:56.483304 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:56.483315 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:56.483317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:56.483833 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:56.484829 (+ 996us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:56.485014 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 10:38:56.485336 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":159,"threads_started":1}
W20260808 10:38:56.489295 21883 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:38:56.489409 21883 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:38:56.489517 21872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:56.490239 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:38:56.490468 21872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:56.496394 21872 catalog_manager.cc:1347] Generated new cluster ID: 934e59022dbe4a31aa37a0d1e4a6ae91
I20260808 10:38:56.496469 21872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:56.501260 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:56.492577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:37245 (local address 127.0.0.1:41599)
0808 10:38:56.492725 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:56.492730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:56.493192 (+ 462us) server_negotiation.cc:407] Connection header received
0808 10:38:56.493350 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:56.493353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:56.493401 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:56.493496 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:56.494434 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:56.495053 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:56.495886 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:56.496121 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:56.496236 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:56.497886 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:56.497889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:56.497892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:38:56.497917 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:56.499438 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:56.499930 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:56.499934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:56.499936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:56.500019 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:38:56.500361 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:38:56.500364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:38:56.500366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:38:56.500536 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:56.500709 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:56.500975 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 10:38:56.501070 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:38:56.503590 21872 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:56.504768 21872 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:56.517498 21872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3eb065e7ac4e4756a47479ce6d30c241: Generated new TSK 0
I20260808 10:38:56.518481 21872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:56.532114 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 21790
W20260808 10:38:56.540782 21775 connection.cc:535] client connection to 127.0.0.1:41599 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41599): Connection reset by peer (error 104)
2026-08-08T10:38:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:56 dist-test-slave-9gcw krb5kdc[21895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:56 dist-test-slave-9gcw krb5kdc[21895](info): set up 2 sockets
Aug 08 10:38:56 dist-test-slave-9gcw krb5kdc[21895](info): commencing operation
krb5kdc: starting...
W20260808 10:38:57.639514 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:57 dist-test-slave-9gcw krb5kdc[21895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:38:57.771812 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:57.891582 21908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:57.891884 21908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:57.891983 21908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:57.896144 21908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:57.896237 21908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:57.896274 21908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:57.896364 21908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:57.896404 21908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:57.901939 21908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:57.903106 21908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:57.904174 21908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:57.913223 21917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:57.913513 21908 server_base.cc:1034] running on GCE node
W20260808 10:38:57.913162 21915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:57.913162 21914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:57.914194 21908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:57.915304 21908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:57.916492 21908 hybrid_clock.cc:648] HybridClock initialized: now 1786185537916475 us; error 31 us; skew 500 ppm
Aug 08 10:38:57 dist-test-slave-9gcw krb5kdc[21895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:57.919358 21908 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:38:57.920223 21908 webserver.cc:460] Webserver started at http://127.0.0.1:36145/ using document root <none> and password file <none>
I20260808 10:38:57.920779 21908 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:57.920883 21908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:57.921126 21908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:57.923230 21908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "f6704e1c29e94a70bc59468ecda0d375"
format_stamp: "Formatted at 2026-08-08 10:38:57 on dist-test-slave-9gcw"
server_key: "c8897efa90c237395fb2e095e8205274"
server_key_iv: "1729619eca2430e07c7457ab13066f8e"
server_key_version: "encryptionkey@0"
I20260808 10:38:57.923812 21908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "f6704e1c29e94a70bc59468ecda0d375"
format_stamp: "Formatted at 2026-08-08 10:38:57 on dist-test-slave-9gcw"
server_key: "c8897efa90c237395fb2e095e8205274"
server_key_iv: "1729619eca2430e07c7457ab13066f8e"
server_key_version: "encryptionkey@0"
I20260808 10:38:57.927511 21908 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 10:38:57.930048 21924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:57.931418 21908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:38:57.931663 21908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "f6704e1c29e94a70bc59468ecda0d375"
format_stamp: "Formatted at 2026-08-08 10:38:57 on dist-test-slave-9gcw"
server_key: "c8897efa90c237395fb2e095e8205274"
server_key_iv: "1729619eca2430e07c7457ab13066f8e"
server_key_version: "encryptionkey@0"
I20260808 10:38:57.931782 21908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:57.951193 21908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:57.954276 21908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:57.954617 21908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:57.962169 21908 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41001
I20260808 10:38:57.962199 21980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41001 every 8 connection(s)
I20260808 10:38:57.963141 21908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:57.965929 21981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:57.969393 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 21908
I20260808 10:38:57.969547 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:57.969851 31306 external_mini_cluster.cc:1428] Setting key e2a354d0bae81d137598cabfc20a785e
I20260808 10:38:57.971885 21981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375: Bootstrap starting.
I20260808 10:38:57.974202 21981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:57.974969 21981 log.cc:824] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:57.977308 21981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375: No bootstrap required, opened a new log
I20260808 10:38:57.979413 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:57.971271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:38237 (local address 127.0.0.1:41001)
0808 10:38:57.971699 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:57.971715 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:57.971749 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:38:57.972445 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:57.972463 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:57.972796 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:57.973125 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:57.973578 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:57.974632 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:57.975782 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:57.976444 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:57.976618 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:57.977503 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:57.977508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:57.977520 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:57.977894 (+ 374us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:57.977948 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:57.977970 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:57.978114 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:57.978321 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 10:38:57.978456 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":120,"threads_started":1}
I20260808 10:38:57.980440 21981 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6704e1c29e94a70bc59468ecda0d375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } }
I20260808 10:38:57.980705 21981 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:57.980764 21981 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6704e1c29e94a70bc59468ecda0d375, State: Initialized, Role: FOLLOWER
I20260808 10:38:57.981231 21981 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6704e1c29e94a70bc59468ecda0d375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } }
I20260808 10:38:57.981416 21981 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:57.981490 21981 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:57.981598 21981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:57.982438 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:38:57.982734 21981 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6704e1c29e94a70bc59468ecda0d375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } }
I20260808 10:38:57.983064 21981 leader_election.cc:302] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6704e1c29e94a70bc59468ecda0d375; no voters:
I20260808 10:38:57.983373 21981 leader_election.cc:288] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:57.983482 21987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:57.983695 21987 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [term 1 LEADER]: Becoming Leader. State: Replica: f6704e1c29e94a70bc59468ecda0d375, State: Running, Role: LEADER
I20260808 10:38:57.984040 21987 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6704e1c29e94a70bc59468ecda0d375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } }
I20260808 10:38:57.984607 21981 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:57.985329 21990 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6704e1c29e94a70bc59468ecda0d375. Latest consensus state: current_term: 1 leader_uuid: "f6704e1c29e94a70bc59468ecda0d375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6704e1c29e94a70bc59468ecda0d375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } } }
I20260808 10:38:57.985471 21990 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:57.985737 21988 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6704e1c29e94a70bc59468ecda0d375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6704e1c29e94a70bc59468ecda0d375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41001 } } }
I20260808 10:38:57.985831 21988 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:57.986332 22000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:57.989488 22000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:57.990684 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:57.984751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:43973 (local address 127.0.0.1:41001)
0808 10:38:57.984867 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:57.984870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:57.985931 (+ 1061us) server_negotiation.cc:407] Connection header received
0808 10:38:57.986118 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:57.986123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:57.986173 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:57.986248 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:57.986672 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:57.987401 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:57.988512 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:38:57.988877 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:38:57.988994 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:38:57.990124 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:57.990128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:57.990131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:57.990144 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:57.990171 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:57.990183 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:57.990244 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:57.990514 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 10:38:57.990552 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:38:57.995343 22000 catalog_manager.cc:1347] Generated new cluster ID: 0db1cfe18f874b8e8f00e73df687db52
I20260808 10:38:57.995406 22000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:58.004902 22000 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:58.006181 22000 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:58.012509 22000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6704e1c29e94a70bc59468ecda0d375: Generated new TSK 0
I20260808 10:38:58.013309 22000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:58.024482 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 21908
W20260808 10:38:58.032218 21893 connection.cc:535] client connection to 127.0.0.1:41001 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41001): Connection reset by peer (error 104)
2026-08-08T10:38:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:58 dist-test-slave-9gcw krb5kdc[22013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:58 dist-test-slave-9gcw krb5kdc[22013](info): set up 2 sockets
Aug 08 10:38:58 dist-test-slave-9gcw krb5kdc[22013](info): commencing operation
krb5kdc: starting...
W20260808 10:38:59.148180 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:38:59 dist-test-slave-9gcw krb5kdc[22013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:38:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:38:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:38:59.283823 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:38:59.399501 22028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:38:59.399770 22028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:38:59.399812 22028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:38:59.403915 22028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:38:59.403980 22028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:38:59.404000 22028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:38:59.404016 22028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:38:59.404031 22028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:38:59.409277 22028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:38:59.410275 22028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:38:59.411230 22028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:38:59.419649 22034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:59.419713 22033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:38:59.419716 22036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:38:59.419739 22028 server_base.cc:1034] running on GCE node
I20260808 10:38:59.420655 22028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:38:59.421813 22028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:38:59.423000 22028 hybrid_clock.cc:648] HybridClock initialized: now 1786185539422972 us; error 42 us; skew 500 ppm
Aug 08 10:38:59 dist-test-slave-9gcw krb5kdc[22013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185539, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:38:59.425793 22028 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:38:59.426708 22028 webserver.cc:460] Webserver started at http://127.0.0.1:40433/ using document root <none> and password file <none>
I20260808 10:38:59.427273 22028 fs_manager.cc:359] Metadata directory not provided
I20260808 10:38:59.427367 22028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:38:59.427604 22028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:38:59.429744 22028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "44c631368c2f4d92bbe1149e759e2781"
format_stamp: "Formatted at 2026-08-08 10:38:59 on dist-test-slave-9gcw"
server_key: "f181b30b9e9488d202e397d890067bcb"
server_key_iv: "a3eb12f3c945c267bd332481639a623e"
server_key_version: "encryptionkey@0"
I20260808 10:38:59.430330 22028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "44c631368c2f4d92bbe1149e759e2781"
format_stamp: "Formatted at 2026-08-08 10:38:59 on dist-test-slave-9gcw"
server_key: "f181b30b9e9488d202e397d890067bcb"
server_key_iv: "a3eb12f3c945c267bd332481639a623e"
server_key_version: "encryptionkey@0"
I20260808 10:38:59.434170 22028 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:38:59.436805 22044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:38:59.438088 22028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 10:38:59.438253 22028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "44c631368c2f4d92bbe1149e759e2781"
format_stamp: "Formatted at 2026-08-08 10:38:59 on dist-test-slave-9gcw"
server_key: "f181b30b9e9488d202e397d890067bcb"
server_key_iv: "a3eb12f3c945c267bd332481639a623e"
server_key_version: "encryptionkey@0"
I20260808 10:38:59.438383 22028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:38:59.443130 22028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:38:59.446135 22028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:38:59.446333 22028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:38:59.453568 22028 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46071
I20260808 10:38:59.453598 22100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46071 every 8 connection(s)
I20260808 10:38:59.454478 22028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:38:59.457201 22101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:38:59.460750 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 22028
I20260808 10:38:59.460891 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:38:59.461218 31306 external_mini_cluster.cc:1428] Setting key dbab9921b4bea2f828c9bdf2ba2c51e1
I20260808 10:38:59.463161 22101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781: Bootstrap starting.
I20260808 10:38:59.465509 22101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781: Neither blocks nor log segments found. Creating new log.
I20260808 10:38:59.466358 22101 log.cc:824] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781: Log is configured to *not* fsync() on all Append() calls
I20260808 10:38:59.467000 22104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:59.462704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:46889 (local address 127.0.0.1:46071)
0808 10:38:59.463169 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:59.463186 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:59.463254 (+ 68us) server_negotiation.cc:407] Connection header received
0808 10:38:59.464006 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:59.464027 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:59.464338 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:59.464715 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:59.465635 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:59.465644 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:59.465661 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:59.465678 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:59.465753 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:59.465776 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:59.465907 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:59.466018 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 10:38:59.466203 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":141,"threads_started":1}
I20260808 10:38:59.468374 22101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781: No bootstrap required, opened a new log
I20260808 10:38:59.470273 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:38:59.472599 22101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c631368c2f4d92bbe1149e759e2781" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } }
I20260808 10:38:59.472931 22101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:38:59.473066 22101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c631368c2f4d92bbe1149e759e2781, State: Initialized, Role: FOLLOWER
I20260808 10:38:59.473817 22101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c631368c2f4d92bbe1149e759e2781" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } }
I20260808 10:38:59.474035 22101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:38:59.474118 22101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:38:59.474232 22101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:38:59.474753 22104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:38:59.472611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:37669 (local address 127.0.0.1:46071)
0808 10:38:59.472782 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:38:59.472787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:38:59.472985 (+ 198us) server_negotiation.cc:407] Connection header received
0808 10:38:59.473110 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:38:59.473116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:38:59.473177 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:38:59.473282 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:38:59.474206 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:38:59.474209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:38:59.474212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:38:59.474224 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:38:59.474257 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:38:59.474271 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:38:59.474370 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 10:38:59.474552 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 10:38:59.474594 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:38:59.475592 22101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c631368c2f4d92bbe1149e759e2781" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } }
I20260808 10:38:59.475977 22101 leader_election.cc:302] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44c631368c2f4d92bbe1149e759e2781; no voters:
I20260808 10:38:59.476334 22101 leader_election.cc:288] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:38:59.476455 22111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:38:59.476693 22111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [term 1 LEADER]: Becoming Leader. State: Replica: 44c631368c2f4d92bbe1149e759e2781, State: Running, Role: LEADER
I20260808 10:38:59.477056 22111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c631368c2f4d92bbe1149e759e2781" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } }
I20260808 10:38:59.477725 22101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:38:59.478863 22112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44c631368c2f4d92bbe1149e759e2781" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c631368c2f4d92bbe1149e759e2781" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } } }
I20260808 10:38:59.478991 22112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:59.479332 22113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c631368c2f4d92bbe1149e759e2781. Latest consensus state: current_term: 1 leader_uuid: "44c631368c2f4d92bbe1149e759e2781" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c631368c2f4d92bbe1149e759e2781" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } } }
I20260808 10:38:59.479435 22113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781 [sys.catalog]: This master's current role is: LEADER
I20260808 10:38:59.480196 22120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:38:59.483462 22120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:38:59.488569 22120 catalog_manager.cc:1347] Generated new cluster ID: b6361608e5ae4e11a29aed430cbf3496
I20260808 10:38:59.488642 22120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:38:59.501180 22120 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:38:59.501497 22120 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:38:59.512212 22120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44c631368c2f4d92bbe1149e759e2781: Generated new TSK 0
I20260808 10:38:59.512941 22120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:38:59.524020 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 22028
2026-08-08T10:38:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:38:59 dist-test-slave-9gcw krb5kdc[22132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:38:59 dist-test-slave-9gcw krb5kdc[22132](info): set up 2 sockets
Aug 08 10:38:59 dist-test-slave-9gcw krb5kdc[22132](info): commencing operation
krb5kdc: starting...
W20260808 10:39:00.659420 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:00 dist-test-slave-9gcw krb5kdc[22132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:39:00.793478 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:00.908385 22147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:00.908721 22147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:00.908808 22147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:00.912947 22147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:00.913054 22147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:00.913091 22147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:00.913131 22147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:00.913177 22147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:00.918632 22147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:00.919744 22147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:00.920807 22147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:00.930109 22154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:00.930147 22153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:00.930193 22156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:00.930382 22147 server_base.cc:1034] running on GCE node
I20260808 10:39:00.931354 22147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:00.932919 22147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:00.934064 22147 hybrid_clock.cc:648] HybridClock initialized: now 1786185540934056 us; error 73 us; skew 500 ppm
Aug 08 10:39:00 dist-test-slave-9gcw krb5kdc[22132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185540, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:00.937328 22147 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:39:00.938264 22147 webserver.cc:460] Webserver started at http://127.0.0.1:46085/ using document root <none> and password file <none>
I20260808 10:39:00.938884 22147 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:00.938948 22147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:00.939188 22147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:00.941461 22147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "fb223474f102457a9b72b709e03da45a"
format_stamp: "Formatted at 2026-08-08 10:39:00 on dist-test-slave-9gcw"
server_key: "4c54e223704c12cd7dab6177857883e2"
server_key_iv: "dfc40dfefaff21211e2559d4b0196870"
server_key_version: "encryptionkey@0"
I20260808 10:39:00.942132 22147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "fb223474f102457a9b72b709e03da45a"
format_stamp: "Formatted at 2026-08-08 10:39:00 on dist-test-slave-9gcw"
server_key: "4c54e223704c12cd7dab6177857883e2"
server_key_iv: "dfc40dfefaff21211e2559d4b0196870"
server_key_version: "encryptionkey@0"
I20260808 10:39:00.945960 22147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:39:00.948375 22163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:00.949765 22147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:39:00.949903 22147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "fb223474f102457a9b72b709e03da45a"
format_stamp: "Formatted at 2026-08-08 10:39:00 on dist-test-slave-9gcw"
server_key: "4c54e223704c12cd7dab6177857883e2"
server_key_iv: "dfc40dfefaff21211e2559d4b0196870"
server_key_version: "encryptionkey@0"
I20260808 10:39:00.950013 22147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:00.975975 22147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:00.978888 22147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:00.979104 22147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:00.986716 22147 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39777
I20260808 10:39:00.986753 22217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39777 every 8 connection(s)
I20260808 10:39:00.987738 22147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:00.990690 22218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:00.991012 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 22147
I20260808 10:39:00.991163 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:00.991488 31306 external_mini_cluster.cc:1428] Setting key 667ec8095a6638e757814b5daf52a9c8
I20260808 10:39:00.998158 22218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a: Bootstrap starting.
I20260808 10:39:01.000828 22218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a: Neither blocks nor log segments found. Creating new log.
Aug 08 10:39:00 dist-test-slave-9gcw krb5kdc[22132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:39:01.001740 22218 log.cc:824] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:01.004331 22218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a: No bootstrap required, opened a new log
I20260808 10:39:01.007623 22221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:00.992955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:38451 (local address 127.0.0.1:39777)
0808 10:39:00.993426 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:00.993446 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:00.993490 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:39:00.994314 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:00.994336 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:00.994785 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:00.995149 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:00.995871 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:00.997420 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:00.998338 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:00.999003 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:00.999156 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:01.001913 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:01.001919 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:01.001935 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:01.001965 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:01.004190 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:01.004776 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:01.004783 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:01.004788 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:01.004930 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:01.005291 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:01.005295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:01.005296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:01.005808 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:01.006065 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:01.006274 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 10:39:01.006508 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":182,"threads_started":1}
I20260808 10:39:01.009208 22218 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223474f102457a9b72b709e03da45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } }
I20260808 10:39:01.009485 22218 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:01.009546 22218 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb223474f102457a9b72b709e03da45a, State: Initialized, Role: FOLLOWER
I20260808 10:39:01.010015 22218 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223474f102457a9b72b709e03da45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } }
I20260808 10:39:01.010185 22218 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:01.010268 22218 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:01.010407 22218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:01.011085 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:01.011672 22218 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223474f102457a9b72b709e03da45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } }
I20260808 10:39:01.012055 22218 leader_election.cc:302] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb223474f102457a9b72b709e03da45a; no voters:
I20260808 10:39:01.012432 22218 leader_election.cc:288] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:01.012923 22224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:01.013216 22224 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [term 1 LEADER]: Becoming Leader. State: Replica: fb223474f102457a9b72b709e03da45a, State: Running, Role: LEADER
I20260808 10:39:01.013530 22224 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223474f102457a9b72b709e03da45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } }
I20260808 10:39:01.013855 22218 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:01.015244 22231 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb223474f102457a9b72b709e03da45a. Latest consensus state: current_term: 1 leader_uuid: "fb223474f102457a9b72b709e03da45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223474f102457a9b72b709e03da45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } } }
I20260808 10:39:01.015394 22231 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:01.015748 22230 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb223474f102457a9b72b709e03da45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223474f102457a9b72b709e03da45a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39777 } } }
I20260808 10:39:01.015844 22230 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:01.016109 22241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:01.019937 22241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:01.022831 22221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:01.013387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:51977 (local address 127.0.0.1:39777)
0808 10:39:01.013495 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:01.013500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:01.013991 (+ 491us) server_negotiation.cc:407] Connection header received
0808 10:39:01.014185 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:01.014190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:01.014247 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:01.014332 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:01.015386 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:01.016177 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:01.017412 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:01.017663 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:01.017780 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:01.019231 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:01.019234 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:01.019237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:01.019273 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:01.020849 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:01.021424 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:01.021427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:01.021429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:01.021514 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:01.021861 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:01.021864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:01.021866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:01.022103 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:01.022250 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:01.022584 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 10:39:01.022685 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:39:01.026343 22241 catalog_manager.cc:1347] Generated new cluster ID: 5509d069e97240d3bf7e13b7a20807fd
I20260808 10:39:01.026427 22241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:01.042142 22241 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:01.043442 22241 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:01.051021 22241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb223474f102457a9b72b709e03da45a: Generated new TSK 0
I20260808 10:39:01.051774 22241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:01.061600 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 22147
W20260808 10:39:01.070271 22130 connection.cc:535] client connection to 127.0.0.1:39777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39777): Connection reset by peer (error 104)
2026-08-08T10:39:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:01 dist-test-slave-9gcw krb5kdc[22252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:01 dist-test-slave-9gcw krb5kdc[22252](info): set up 2 sockets
Aug 08 10:39:01 dist-test-slave-9gcw krb5kdc[22252](info): commencing operation
krb5kdc: starting...
W20260808 10:39:02.219079 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:02 dist-test-slave-9gcw krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:39:02.350373 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44007
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:02.464466 22266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:02.464789 22266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:02.464901 22266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:02.469038 22266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:02.469149 22266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:02.469209 22266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:02.469252 22266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:02.469300 22266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:02.474992 22266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44007
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:02.476249 22266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:02.477350 22266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:02.486189 22272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:02.486204 22271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:02.486296 22274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:02.486644 22266 server_base.cc:1034] running on GCE node
I20260808 10:39:02.487226 22266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:02.488409 22266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:02.489596 22266 hybrid_clock.cc:648] HybridClock initialized: now 1786185542489577 us; error 31 us; skew 500 ppm
Aug 08 10:39:02 dist-test-slave-9gcw krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185542, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:02.498605 22266 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:39:02.499572 22266 webserver.cc:460] Webserver started at http://127.0.0.1:40841/ using document root <none> and password file <none>
I20260808 10:39:02.500169 22266 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:02.500265 22266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:02.500504 22266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:02.502737 22266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "43fdca46053c4b61a2e08f730d06fcb4"
format_stamp: "Formatted at 2026-08-08 10:39:02 on dist-test-slave-9gcw"
server_key: "f09b6eaae94395cbd31fc8e880e20159"
server_key_iv: "6d69f539a0b40b832ad52f9bcb8e6ad3"
server_key_version: "encryptionkey@0"
I20260808 10:39:02.503340 22266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "43fdca46053c4b61a2e08f730d06fcb4"
format_stamp: "Formatted at 2026-08-08 10:39:02 on dist-test-slave-9gcw"
server_key: "f09b6eaae94395cbd31fc8e880e20159"
server_key_iv: "6d69f539a0b40b832ad52f9bcb8e6ad3"
server_key_version: "encryptionkey@0"
I20260808 10:39:02.507396 22266 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:02.510033 22281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:02.511373 22266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:39:02.511531 22266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "43fdca46053c4b61a2e08f730d06fcb4"
format_stamp: "Formatted at 2026-08-08 10:39:02 on dist-test-slave-9gcw"
server_key: "f09b6eaae94395cbd31fc8e880e20159"
server_key_iv: "6d69f539a0b40b832ad52f9bcb8e6ad3"
server_key_version: "encryptionkey@0"
I20260808 10:39:02.511653 22266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:02.533861 22266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:02.536926 22266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:02.537213 22266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:02.544670 22266 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44007
I20260808 10:39:02.544679 22340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44007 every 8 connection(s)
I20260808 10:39:02.545686 22266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:02.547694 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 22266
I20260808 10:39:02.547876 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:02.548184 31306 external_mini_cluster.cc:1428] Setting key dab14480c369bfe1f935e2c2aac82b73
I20260808 10:39:02.548765 22341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:02.555609 22341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4: Bootstrap starting.
I20260808 10:39:02.557546 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:02.549598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:41253 (local address 127.0.0.1:44007)
0808 10:39:02.550035 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:02.550061 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:02.550096 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:39:02.550739 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:02.550755 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:02.551082 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:02.551355 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:02.551779 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:02.552874 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:02.554096 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:02.554572 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:02.554699 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:02.555631 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:02.555635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:02.555731 (+ 96us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:02.556143 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:02.556203 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:02.556227 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:02.556329 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:02.556532 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 10:39:02.556775 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":151,"threads_started":1}
I20260808 10:39:02.558010 22341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:02.558838 22341 log.cc:824] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:02.560751 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:02.561013 22341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4: No bootstrap required, opened a new log
I20260808 10:39:02.565845 22341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fdca46053c4b61a2e08f730d06fcb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44007 } }
I20260808 10:39:02.566147 22341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:02.566215 22341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43fdca46053c4b61a2e08f730d06fcb4, State: Initialized, Role: FOLLOWER
I20260808 10:39:02.566746 22341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fdca46053c4b61a2e08f730d06fcb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44007 } }
I20260808 10:39:02.566936 22341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:02.567018 22341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:02.567134 22341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:02.567799 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:02.563100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:60291 (local address 127.0.0.1:44007)
0808 10:39:02.563240 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:02.563244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:02.563431 (+ 187us) server_negotiation.cc:407] Connection header received
0808 10:39:02.563522 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:02.563526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:02.563583 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:02.563688 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:02.563974 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:02.564775 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:02.565678 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:02.566009 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:02.566125 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:02.567143 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:02.567149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:02.567152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:02.567168 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:02.567198 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:02.567212 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:02.567307 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:02.567579 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 10:39:02.567630 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:39:02.568418 22341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fdca46053c4b61a2e08f730d06fcb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44007 } }
I20260808 10:39:02.568781 22341 leader_election.cc:302] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43fdca46053c4b61a2e08f730d06fcb4; no voters:
I20260808 10:39:02.569125 22341 leader_election.cc:288] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:02.569234 22353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:02.569439 22353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [term 1 LEADER]: Becoming Leader. State: Replica: 43fdca46053c4b61a2e08f730d06fcb4, State: Running, Role: LEADER
I20260808 10:39:02.569746 22353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fdca46053c4b61a2e08f730d06fcb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44007 } }
I20260808 10:39:02.570500 22341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:02.571583 22354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43fdca46053c4b61a2e08f730d06fcb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fdca46053c4b61a2e08f730d06fcb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44007 } } }
I20260808 10:39:02.571733 22354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:02.572037 22355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43fdca46053c4b61a2e08f730d06fcb4. Latest consensus state: current_term: 1 leader_uuid: "43fdca46053c4b61a2e08f730d06fcb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fdca46053c4b61a2e08f730d06fcb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44007 } } }
I20260808 10:39:02.572129 22355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4 [sys.catalog]: This master's current role is: LEADER
W20260808 10:39:02.575676 22368 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:02.575768 22368 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:02.575840 22362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:02.576508 22362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:02.581765 22362 catalog_manager.cc:1347] Generated new cluster ID: b1d4f72ba1df4d0eb0c77b7e1654c807
I20260808 10:39:02.581856 22362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:02.600860 22362 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:02.602151 22362 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:02.609614 22362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43fdca46053c4b61a2e08f730d06fcb4: Generated new TSK 0
I20260808 10:39:02.610466 22362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:02.614970 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 22266
W20260808 10:39:02.624164 22250 connection.cc:535] client connection to 127.0.0.1:44007 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44007): Connection reset by peer (error 104)
2026-08-08T10:39:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:02 dist-test-slave-9gcw krb5kdc[22374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:02 dist-test-slave-9gcw krb5kdc[22374](info): set up 2 sockets
Aug 08 10:39:02 dist-test-slave-9gcw krb5kdc[22374](info): commencing operation
krb5kdc: starting...
W20260808 10:39:03.764278 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:03 dist-test-slave-9gcw krb5kdc[22374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:39:03.901573 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40763
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:04.018023 22388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:04.018366 22388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:04.018527 22388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:04.022632 22388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:04.022724 22388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:04.022763 22388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:04.022815 22388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:04.022871 22388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:04.028281 22388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40763
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:04.029498 22388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:04.030560 22388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:04.039700 22394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:04.039777 22396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:04.039700 22393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:04.040081 22388 server_base.cc:1034] running on GCE node
I20260808 10:39:04.040766 22388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:04.042026 22388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:04.043232 22388 hybrid_clock.cc:648] HybridClock initialized: now 1786185544043216 us; error 29 us; skew 500 ppm
Aug 08 10:39:04 dist-test-slave-9gcw krb5kdc[22374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185544, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:04.046182 22388 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:39:04.047185 22388 webserver.cc:460] Webserver started at http://127.0.0.1:34741/ using document root <none> and password file <none>
I20260808 10:39:04.047773 22388 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:04.047869 22388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:04.048110 22388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:04.050307 22388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "8cbf49336d4340989ae0cc382c33b5e0"
format_stamp: "Formatted at 2026-08-08 10:39:04 on dist-test-slave-9gcw"
server_key: "6704328976581f2972c8ad57996ebb05"
server_key_iv: "c142e0bf601e82538352f32ebb2af774"
server_key_version: "encryptionkey@0"
I20260808 10:39:04.050894 22388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "8cbf49336d4340989ae0cc382c33b5e0"
format_stamp: "Formatted at 2026-08-08 10:39:04 on dist-test-slave-9gcw"
server_key: "6704328976581f2972c8ad57996ebb05"
server_key_iv: "c142e0bf601e82538352f32ebb2af774"
server_key_version: "encryptionkey@0"
I20260808 10:39:04.054757 22388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:39:04.057301 22403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:04.058595 22388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:39:04.058744 22388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "8cbf49336d4340989ae0cc382c33b5e0"
format_stamp: "Formatted at 2026-08-08 10:39:04 on dist-test-slave-9gcw"
server_key: "6704328976581f2972c8ad57996ebb05"
server_key_iv: "c142e0bf601e82538352f32ebb2af774"
server_key_version: "encryptionkey@0"
I20260808 10:39:04.058866 22388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:04.063560 22388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:04.066520 22388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:04.066721 22388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:04.074019 22388 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40763
I20260808 10:39:04.074048 22459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40763 every 8 connection(s)
I20260808 10:39:04.074987 22388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:04.077898 22461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:04.079351 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 22388
I20260808 10:39:04.079504 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:04.079838 31306 external_mini_cluster.cc:1428] Setting key 4d2e18a35c72350358e2877db344912f
I20260808 10:39:04.084486 22461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0: Bootstrap starting.
I20260808 10:39:04.085471 22465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:04.081477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:37945 (local address 127.0.0.1:40763)
0808 10:39:04.081937 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:04.081952 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:04.081988 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:39:04.082684 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:04.082702 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:04.082985 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:04.083329 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:04.084132 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:04.084136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:04.084148 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:04.084158 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:04.084214 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:04.084233 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:04.084419 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:04.084492 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 10:39:04.084658 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":144,"threads_started":1}
I20260808 10:39:04.087165 22461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:04.088081 22461 log.cc:824] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:04.088456 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:04.090595 22461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0: No bootstrap required, opened a new log
I20260808 10:39:04.092418 22465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:04.090744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:50633 (local address 127.0.0.1:40763)
0808 10:39:04.090884 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:04.090889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:04.091102 (+ 213us) server_negotiation.cc:407] Connection header received
0808 10:39:04.091178 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:04.091181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:04.091217 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:04.091316 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:04.092029 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:04.092032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:04.092035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:04.092048 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:04.092077 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:04.092090 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:04.092175 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:04.092243 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 10:39:04.092283 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:39:04.095160 22461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbf49336d4340989ae0cc382c33b5e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260808 10:39:04.095418 22461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:04.095499 22461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cbf49336d4340989ae0cc382c33b5e0, State: Initialized, Role: FOLLOWER
I20260808 10:39:04.095937 22461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbf49336d4340989ae0cc382c33b5e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260808 10:39:04.096091 22461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:04.096160 22461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:04.096282 22461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:04.097482 22461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbf49336d4340989ae0cc382c33b5e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260808 10:39:04.097889 22461 leader_election.cc:302] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cbf49336d4340989ae0cc382c33b5e0; no voters:
I20260808 10:39:04.098349 22461 leader_election.cc:288] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:04.098492 22472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:04.098798 22472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [term 1 LEADER]: Becoming Leader. State: Replica: 8cbf49336d4340989ae0cc382c33b5e0, State: Running, Role: LEADER
I20260808 10:39:04.099215 22472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbf49336d4340989ae0cc382c33b5e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260808 10:39:04.099781 22461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:04.101234 22473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cbf49336d4340989ae0cc382c33b5e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbf49336d4340989ae0cc382c33b5e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } } }
I20260808 10:39:04.101379 22473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:04.101626 22474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cbf49336d4340989ae0cc382c33b5e0. Latest consensus state: current_term: 1 leader_uuid: "8cbf49336d4340989ae0cc382c33b5e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cbf49336d4340989ae0cc382c33b5e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } } }
I20260808 10:39:04.101734 22474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0 [sys.catalog]: This master's current role is: LEADER
W20260808 10:39:04.105011 22487 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:04.105106 22487 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:04.105235 22482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:04.106165 22482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:04.111943 22482 catalog_manager.cc:1347] Generated new cluster ID: ca2d32357bb94cdd8f230a079031653c
I20260808 10:39:04.112020 22482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:04.139753 22482 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:04.140178 22482 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:04.146135 22482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cbf49336d4340989ae0cc382c33b5e0: Generated new TSK 0
I20260808 10:39:04.147059 22482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:04.151335 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 22388
2026-08-08T10:39:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:04 dist-test-slave-9gcw krb5kdc[22493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:04 dist-test-slave-9gcw krb5kdc[22493](info): set up 2 sockets
Aug 08 10:39:04 dist-test-slave-9gcw krb5kdc[22493](info): commencing operation
krb5kdc: starting...
W20260808 10:39:05.255944 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:05 dist-test-slave-9gcw krb5kdc[22493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:39:05.388063 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:38697
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:33403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:38697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:05.509126 22506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:05.509445 22506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:05.509492 22506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:05.513558 22506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:05.513638 22506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:05.513661 22506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:05.513680 22506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:05.513700 22506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:05.519277 22506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:33403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:38697
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:38697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:05.520375 22506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:05.521413 22506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:05.530467 22512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:05.530464 22514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:05.530520 22511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:05.530824 22506 server_base.cc:1034] running on GCE node
I20260808 10:39:05.531389 22506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:05.532546 22506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:05.533731 22506 hybrid_clock.cc:648] HybridClock initialized: now 1786185545533700 us; error 48 us; skew 500 ppm
Aug 08 10:39:05 dist-test-slave-9gcw krb5kdc[22493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185545, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:05.536561 22506 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:39:05.537601 22506 webserver.cc:460] Webserver started at http://127.30.146.190:40245/ using document root <none> and password file <none>
I20260808 10:39:05.538190 22506 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:05.538297 22506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:05.538552 22506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:05.540671 22506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "06008634a315428a866756b3ca5b5006"
format_stamp: "Formatted at 2026-08-08 10:39:05 on dist-test-slave-9gcw"
server_key: "aec7e3330e7009c1e22f2ce5b35d121c"
server_key_iv: "a8a6bb63411b5003c4d095b2e394316a"
server_key_version: "encryptionkey@0"
I20260808 10:39:05.541342 22506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "06008634a315428a866756b3ca5b5006"
format_stamp: "Formatted at 2026-08-08 10:39:05 on dist-test-slave-9gcw"
server_key: "aec7e3330e7009c1e22f2ce5b35d121c"
server_key_iv: "a8a6bb63411b5003c4d095b2e394316a"
server_key_version: "encryptionkey@0"
I20260808 10:39:05.545312 22506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:39:05.547845 22522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:05.549126 22506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:05.549341 22506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "06008634a315428a866756b3ca5b5006"
format_stamp: "Formatted at 2026-08-08 10:39:05 on dist-test-slave-9gcw"
server_key: "aec7e3330e7009c1e22f2ce5b35d121c"
server_key_iv: "a8a6bb63411b5003c4d095b2e394316a"
server_key_version: "encryptionkey@0"
I20260808 10:39:05.549467 22506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:05.577987 22506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:05.581203 22506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:05.581436 22506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:05.589242 22506 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:38697
I20260808 10:39:05.589269 22578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:38697 every 8 connection(s)
I20260808 10:39:05.590222 22506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:05.593026 22579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:05.595927 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 22506
I20260808 10:39:05.596069 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:05.596359 31306 external_mini_cluster.cc:1428] Setting key 84edc919245a23ebc80506cf99773836
I20260808 10:39:05.599097 22579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006: Bootstrap starting.
I20260808 10:39:05.601346 22579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:05.602084 22579 log.cc:824] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:05.604331 22579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006: No bootstrap required, opened a new log
Aug 08 10:39:05 dist-test-slave-9gcw krb5kdc[22493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:39:05.608422 22579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06008634a315428a866756b3ca5b5006" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 38697 } }
I20260808 10:39:05.608713 22579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:05.608763 22579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06008634a315428a866756b3ca5b5006, State: Initialized, Role: FOLLOWER
I20260808 10:39:05.609311 22579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06008634a315428a866756b3ca5b5006" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 38697 } }
I20260808 10:39:05.609485 22579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:05.609577 22579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:05.609700 22579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:05.611016 22579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06008634a315428a866756b3ca5b5006" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 38697 } }
I20260808 10:39:05.611406 22579 leader_election.cc:302] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06008634a315428a866756b3ca5b5006; no voters:
I20260808 10:39:05.611779 22579 leader_election.cc:288] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:05.611904 22584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:05.612207 22584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [term 1 LEADER]: Becoming Leader. State: Replica: 06008634a315428a866756b3ca5b5006, State: Running, Role: LEADER
I20260808 10:39:05.612624 22584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06008634a315428a866756b3ca5b5006" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 38697 } }
I20260808 10:39:05.613183 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:05.597863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60423 (local address 127.30.146.190:38697)
0808 10:39:05.598346 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:05.598361 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:05.598398 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:39:05.599074 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:05.599091 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:05.599493 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:05.599840 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:05.600658 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:05.601771 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:05.602816 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:05.603498 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:05.603669 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:05.606449 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:05.606455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:05.606472 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:05.606503 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:05.609230 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:05.609986 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:05.609995 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:05.610003 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:05.610183 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:05.610695 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:05.610698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:05.610700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:05.611286 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:05.611515 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:05.611894 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 10:39:05.612145 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":144,"threads_started":1}
I20260808 10:39:05.613250 22579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:05.614617 22585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06008634a315428a866756b3ca5b5006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06008634a315428a866756b3ca5b5006" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 38697 } } }
I20260808 10:39:05.614598 22586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06008634a315428a866756b3ca5b5006. Latest consensus state: current_term: 1 leader_uuid: "06008634a315428a866756b3ca5b5006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06008634a315428a866756b3ca5b5006" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 38697 } } }
I20260808 10:39:05.614753 22585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:05.614753 22586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:05.615206 22591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:05.616606 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:05.618651 22591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:05.624763 22591 catalog_manager.cc:1347] Generated new cluster ID: 42893b2be33a4ddcb4cb22c97ea55901
I20260808 10:39:05.624847 22591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:05.631564 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:05.618922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34077 (local address 127.30.146.190:38697)
0808 10:39:05.619033 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:05.619037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:05.621425 (+ 2388us) server_negotiation.cc:407] Connection header received
0808 10:39:05.621662 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:05.621666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:05.621716 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:05.621779 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:05.622823 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:05.623483 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:05.624734 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:05.625106 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:05.625387 (+ 281us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:05.627371 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:05.627375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:05.627471 (+ 96us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:05.627502 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:05.629304 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:05.629951 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:05.629955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:05.629957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:05.630046 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:05.630451 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:05.630457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:05.630461 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:05.630700 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:05.630836 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:05.631287 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 10:39:05.631393 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:39:05.634166 22591 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:05.635073 22591 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:05.646169 22591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06008634a315428a866756b3ca5b5006: Generated new TSK 0
I20260808 10:39:05.646981 22591 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:05.655283 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 22506
W20260808 10:39:05.663991 22491 connection.cc:535] client connection to 127.30.146.190:38697 recv error: Network error: failed to read from TLS socket (remote: 127.30.146.190:38697): Connection reset by peer (error 104)
2026-08-08T10:39:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:05 dist-test-slave-9gcw krb5kdc[22613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:05 dist-test-slave-9gcw krb5kdc[22613](info): set up 2 sockets
Aug 08 10:39:05 dist-test-slave-9gcw krb5kdc[22613](info): commencing operation
krb5kdc: starting...
W20260808 10:39:06.755954 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:06 dist-test-slave-9gcw krb5kdc[22613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:39:06.894093 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:42625
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:35045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:42625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:07.013106 22629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:07.013398 22629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:07.013440 22629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:07.017589 22629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:07.017652 22629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:07.017673 22629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:07.017689 22629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:07.017705 22629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:07.023051 22629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:35045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:42625
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:42625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:07.024150 22629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:07.025223 22629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:39:07.034051 22629 server_base.cc:1034] running on GCE node
W20260808 10:39:07.033985 22638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:07.033993 22634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:07.034224 22636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:07.034924 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:07.036099 22629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:07.037292 22629 hybrid_clock.cc:648] HybridClock initialized: now 1786185547037264 us; error 41 us; skew 500 ppm
Aug 08 10:39:07 dist-test-slave-9gcw krb5kdc[22613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185547, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:07.040091 22629 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:39:07.041285 22629 webserver.cc:460] Webserver started at http://127.30.146.190:33927/ using document root <none> and password file <none>
I20260808 10:39:07.041882 22629 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:07.041978 22629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:07.042200 22629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:07.044353 22629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "8c027b1821624aee80751aafa6d4e96c"
format_stamp: "Formatted at 2026-08-08 10:39:07 on dist-test-slave-9gcw"
server_key: "23ac92341c3b0f08a90ee42266cb0cce"
server_key_iv: "9e6623424bdac0399a370097e74da8fb"
server_key_version: "encryptionkey@0"
I20260808 10:39:07.044942 22629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "8c027b1821624aee80751aafa6d4e96c"
format_stamp: "Formatted at 2026-08-08 10:39:07 on dist-test-slave-9gcw"
server_key: "23ac92341c3b0f08a90ee42266cb0cce"
server_key_iv: "9e6623424bdac0399a370097e74da8fb"
server_key_version: "encryptionkey@0"
I20260808 10:39:07.048831 22629 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 10:39:07.051407 22645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:07.052618 22629 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:07.052767 22629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "8c027b1821624aee80751aafa6d4e96c"
format_stamp: "Formatted at 2026-08-08 10:39:07 on dist-test-slave-9gcw"
server_key: "23ac92341c3b0f08a90ee42266cb0cce"
server_key_iv: "9e6623424bdac0399a370097e74da8fb"
server_key_version: "encryptionkey@0"
I20260808 10:39:07.052886 22629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:07.082275 22629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:07.085799 22629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:07.086031 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:07.093859 22629 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:42625
I20260808 10:39:07.093868 22701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:42625 every 8 connection(s)
I20260808 10:39:07.094892 22629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:07.097781 22702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:07.102214 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 22629
I20260808 10:39:07.102411 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:07.102710 31306 external_mini_cluster.cc:1428] Setting key 0986b81e361125228324ce084ce126e4
I20260808 10:39:07.103798 22702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c: Bootstrap starting.
I20260808 10:39:07.106683 22702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:07.107585 22702 log.cc:824] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:07.110050 22702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c: No bootstrap required, opened a new log
I20260808 10:39:07.112478 22705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:07.104248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35853 (local address 127.30.146.190:42625)
0808 10:39:07.104702 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:07.104718 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:07.104755 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:39:07.105505 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:07.105523 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:07.105897 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:07.106185 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:07.106576 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:07.107849 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:07.108997 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:07.109553 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:07.109680 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:07.110689 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:07.110694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:07.110708 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:07.111109 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:07.111171 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:07.111202 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:07.111315 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:07.111521 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 10:39:07.111690 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":152,"threads_started":1}
I20260808 10:39:07.114336 22702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c027b1821624aee80751aafa6d4e96c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42625 } }
I20260808 10:39:07.114600 22702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:07.114725 22702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c027b1821624aee80751aafa6d4e96c, State: Initialized, Role: FOLLOWER
I20260808 10:39:07.115139 22702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c027b1821624aee80751aafa6d4e96c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42625 } }
I20260808 10:39:07.115337 22702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:07.115406 22702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:07.115505 22702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:07.115708 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:07.116834 22702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c027b1821624aee80751aafa6d4e96c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42625 } }
I20260808 10:39:07.117292 22702 leader_election.cc:302] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c027b1821624aee80751aafa6d4e96c; no voters:
I20260808 10:39:07.117681 22702 leader_election.cc:288] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:07.117791 22712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:07.118131 22712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [term 1 LEADER]: Becoming Leader. State: Replica: 8c027b1821624aee80751aafa6d4e96c, State: Running, Role: LEADER
I20260808 10:39:07.118546 22712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c027b1821624aee80751aafa6d4e96c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42625 } }
I20260808 10:39:07.119058 22702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:07.119726 22712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c027b1821624aee80751aafa6d4e96c. Latest consensus state: current_term: 1 leader_uuid: "8c027b1821624aee80751aafa6d4e96c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c027b1821624aee80751aafa6d4e96c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42625 } } }
I20260808 10:39:07.119905 22712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:07.120208 22715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c027b1821624aee80751aafa6d4e96c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c027b1821624aee80751aafa6d4e96c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 42625 } } }
I20260808 10:39:07.120355 22715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:07.120461 22722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:07.123867 22722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:07.126523 22705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:07.118188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38397 (local address 127.30.146.190:42625)
0808 10:39:07.118291 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:07.118296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:07.121650 (+ 3354us) server_negotiation.cc:407] Connection header received
0808 10:39:07.121807 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:07.121811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:07.121884 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:07.121969 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:07.122449 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:07.123249 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:07.124407 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:07.124744 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:07.124822 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:07.125980 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:07.125983 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:07.125987 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:07.126004 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:07.126034 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:07.126048 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:07.126146 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:07.126330 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 10:39:07.126366 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:39:07.130453 22722 catalog_manager.cc:1347] Generated new cluster ID: 172db77d39d54ceaaa1e1ee3855e5600
I20260808 10:39:07.130519 22722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:07.148365 22722 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:07.149712 22722 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:07.166631 22722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c027b1821624aee80751aafa6d4e96c: Generated new TSK 0
I20260808 10:39:07.167552 22722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:07.172617 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 22629
W20260808 10:39:07.180001 22611 connection.cc:535] client connection to 127.30.146.190:42625 recv error: Network error: failed to read from TLS socket (remote: 127.30.146.190:42625): Connection reset by peer (error 104)
2026-08-08T10:39:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:07 dist-test-slave-9gcw krb5kdc[22737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:07 dist-test-slave-9gcw krb5kdc[22737](info): set up 2 sockets
Aug 08 10:39:07 dist-test-slave-9gcw krb5kdc[22737](info): commencing operation
krb5kdc: starting...
W20260808 10:39:08.279788 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:08 dist-test-slave-9gcw krb5kdc[22737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:39:08.411890 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33607
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:33607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:08.528662 22751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:08.528926 22751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:08.528968 22751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:08.533025 22751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:08.533094 22751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:08.533116 22751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:08.533197 22751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:08.533234 22751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:08.538702 22751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:33607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:33607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:08.539776 22751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:08.540879 22751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:08.549706 22756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:08.549685 22757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:08.549847 22759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:08.549870 22751 server_base.cc:1034] running on GCE node
I20260808 10:39:08.550623 22751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:08.551800 22751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:08.553017 22751 hybrid_clock.cc:648] HybridClock initialized: now 1786185548553006 us; error 49 us; skew 500 ppm
Aug 08 10:39:08 dist-test-slave-9gcw krb5kdc[22737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185548, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:08.559664 22751 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:39:08.560720 22751 webserver.cc:460] Webserver started at http://127.30.146.190:33119/ using document root <none> and password file <none>
I20260808 10:39:08.561399 22751 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:08.561498 22751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:08.561743 22751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:08.563891 22751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "c4dcdbc4b44440bfb1244ddc27560ff3"
format_stamp: "Formatted at 2026-08-08 10:39:08 on dist-test-slave-9gcw"
server_key: "c54197f47e00778db282dfcbe2499688"
server_key_iv: "dcb078d43663553d54628e5b8c036d28"
server_key_version: "encryptionkey@0"
I20260808 10:39:08.564483 22751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "c4dcdbc4b44440bfb1244ddc27560ff3"
format_stamp: "Formatted at 2026-08-08 10:39:08 on dist-test-slave-9gcw"
server_key: "c54197f47e00778db282dfcbe2499688"
server_key_iv: "dcb078d43663553d54628e5b8c036d28"
server_key_version: "encryptionkey@0"
I20260808 10:39:08.568253 22751 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:39:08.570744 22766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:08.571992 22751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:08.572136 22751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "c4dcdbc4b44440bfb1244ddc27560ff3"
format_stamp: "Formatted at 2026-08-08 10:39:08 on dist-test-slave-9gcw"
server_key: "c54197f47e00778db282dfcbe2499688"
server_key_iv: "dcb078d43663553d54628e5b8c036d28"
server_key_version: "encryptionkey@0"
I20260808 10:39:08.572259 22751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:08.577024 22751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:08.579939 22751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:08.580207 22751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:08.587881 22751 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:33607
I20260808 10:39:08.587946 22823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:33607 every 8 connection(s)
I20260808 10:39:08.588878 22751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:08.589243 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 22751
I20260808 10:39:08.589407 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:08.589767 31306 external_mini_cluster.cc:1428] Setting key ef6bbdde542a5da798a8f5e1c863bca2
I20260808 10:39:08.592257 22824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:08.595034 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:08.591191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50417 (local address 127.30.146.190:33607)
0808 10:39:08.591657 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:08.591672 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:08.591708 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:39:08.592434 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:08.592452 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:08.592738 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:08.593071 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:08.593879 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:08.593883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:08.593895 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:08.593905 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:08.593956 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:08.593974 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:08.594126 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:08.594193 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 10:39:08.594356 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":160,"threads_started":1}
I20260808 10:39:08.597909 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:08.598762 22824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3: Bootstrap starting.
I20260808 10:39:08.601502 22824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:08.601802 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:08.600093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32879 (local address 127.30.146.190:33607)
0808 10:39:08.600212 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:08.600216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:08.600390 (+ 174us) server_negotiation.cc:407] Connection header received
0808 10:39:08.600506 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:08.600508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:08.600537 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:08.600601 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:08.601339 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:08.601342 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:08.601344 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:08.601354 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:08.601378 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:08.601388 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:08.601457 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:08.601607 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 10:39:08.601647 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:39:08.602368 22824 log.cc:824] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:08.604346 22824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3: No bootstrap required, opened a new log
I20260808 10:39:08.607893 22824 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4dcdbc4b44440bfb1244ddc27560ff3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33607 } }
I20260808 10:39:08.608155 22824 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:08.608220 22824 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4dcdbc4b44440bfb1244ddc27560ff3, State: Initialized, Role: FOLLOWER
I20260808 10:39:08.608692 22824 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4dcdbc4b44440bfb1244ddc27560ff3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33607 } }
I20260808 10:39:08.608856 22824 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:08.608932 22824 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:08.609053 22824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:08.610033 22824 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4dcdbc4b44440bfb1244ddc27560ff3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33607 } }
I20260808 10:39:08.610381 22824 leader_election.cc:302] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4dcdbc4b44440bfb1244ddc27560ff3; no voters:
I20260808 10:39:08.610710 22824 leader_election.cc:288] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:08.610898 22835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:08.611155 22835 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [term 1 LEADER]: Becoming Leader. State: Replica: c4dcdbc4b44440bfb1244ddc27560ff3, State: Running, Role: LEADER
I20260808 10:39:08.611459 22835 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4dcdbc4b44440bfb1244ddc27560ff3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33607 } }
I20260808 10:39:08.612092 22824 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:08.613289 22836 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4dcdbc4b44440bfb1244ddc27560ff3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4dcdbc4b44440bfb1244ddc27560ff3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33607 } } }
I20260808 10:39:08.613420 22836 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:08.613698 22837 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4dcdbc4b44440bfb1244ddc27560ff3. Latest consensus state: current_term: 1 leader_uuid: "c4dcdbc4b44440bfb1244ddc27560ff3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4dcdbc4b44440bfb1244ddc27560ff3" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 33607 } } }
I20260808 10:39:08.613796 22837 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:08.613822 22844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:08.617067 22844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:08.623093 22844 catalog_manager.cc:1347] Generated new cluster ID: 6ae8d66bb81744ed86e2901e01559224
I20260808 10:39:08.623190 22844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:08.646413 22844 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:08.646819 22844 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:08.659430 22844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4dcdbc4b44440bfb1244ddc27560ff3: Generated new TSK 0
I20260808 10:39:08.660295 22844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:08.666870 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 22751
2026-08-08T10:39:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:08 dist-test-slave-9gcw krb5kdc[22859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:08 dist-test-slave-9gcw krb5kdc[22859](info): set up 2 sockets
Aug 08 10:39:08 dist-test-slave-9gcw krb5kdc[22859](info): commencing operation
krb5kdc: starting...
W20260808 10:39:09.772583 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:09 dist-test-slave-9gcw krb5kdc[22859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:39:09.908959 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:44709
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:10.026173 22874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:10.026432 22874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:10.026474 22874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:10.030550 22874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:10.030612 22874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:10.030632 22874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:10.030656 22874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:10.030712 22874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:10.036121 22874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:44709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:10.037300 22874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:10.038383 22874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:10.047168 22882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:10.047178 22880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:10.047178 22879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:10.047659 22874 server_base.cc:1034] running on GCE node
I20260808 10:39:10.048269 22874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:10.049464 22874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:10.050629 22874 hybrid_clock.cc:648] HybridClock initialized: now 1786185550050596 us; error 48 us; skew 500 ppm
Aug 08 10:39:10 dist-test-slave-9gcw krb5kdc[22859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185550, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:10.053570 22874 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:39:10.054575 22874 webserver.cc:460] Webserver started at http://127.30.146.190:42973/ using document root <none> and password file <none>
I20260808 10:39:10.055145 22874 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:10.055209 22874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:10.055455 22874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:10.057664 22874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "261691cb05f84358a06b9299a05d0ae0"
format_stamp: "Formatted at 2026-08-08 10:39:10 on dist-test-slave-9gcw"
server_key: "629ee0db900df935bca15d22a25fec8a"
server_key_iv: "a9be1c408575253ed6fc3ab8a40f0f10"
server_key_version: "encryptionkey@0"
I20260808 10:39:10.058239 22874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "261691cb05f84358a06b9299a05d0ae0"
format_stamp: "Formatted at 2026-08-08 10:39:10 on dist-test-slave-9gcw"
server_key: "629ee0db900df935bca15d22a25fec8a"
server_key_iv: "a9be1c408575253ed6fc3ab8a40f0f10"
server_key_version: "encryptionkey@0"
I20260808 10:39:10.062000 22874 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:39:10.064474 22891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:10.065809 22874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:39:10.065958 22874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "261691cb05f84358a06b9299a05d0ae0"
format_stamp: "Formatted at 2026-08-08 10:39:10 on dist-test-slave-9gcw"
server_key: "629ee0db900df935bca15d22a25fec8a"
server_key_iv: "a9be1c408575253ed6fc3ab8a40f0f10"
server_key_version: "encryptionkey@0"
I20260808 10:39:10.066084 22874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:10.085621 22874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:10.089448 22874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:10.089676 22874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:10.097318 22874 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:44709
I20260808 10:39:10.097342 22946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:44709 every 8 connection(s)
I20260808 10:39:10.098330 22874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:10.101228 22947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:10.106932 22947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0: Bootstrap starting.
I20260808 10:39:10.107195 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 22874
I20260808 10:39:10.107364 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:10.107666 31306 external_mini_cluster.cc:1428] Setting key 48b4caf1ba27d31f968b77088875c6a0
I20260808 10:39:10.109889 22947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:10.110831 22947 log.cc:824] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:10.113880 22947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0: No bootstrap required, opened a new log
Aug 08 10:39:10 dist-test-slave-9gcw krb5kdc[22859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:39:10.119174 22947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261691cb05f84358a06b9299a05d0ae0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44709 } }
I20260808 10:39:10.119489 22947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:10.119604 22947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 261691cb05f84358a06b9299a05d0ae0, State: Initialized, Role: FOLLOWER
I20260808 10:39:10.120146 22947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261691cb05f84358a06b9299a05d0ae0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44709 } }
I20260808 10:39:10.120286 22947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:10.120390 22947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:10.120524 22947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:10.121670 22947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261691cb05f84358a06b9299a05d0ae0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44709 } }
I20260808 10:39:10.122046 22947 leader_election.cc:302] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 261691cb05f84358a06b9299a05d0ae0; no voters:
I20260808 10:39:10.122466 22947 leader_election.cc:288] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:10.122522 22954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:10.122879 22954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [term 1 LEADER]: Becoming Leader. State: Replica: 261691cb05f84358a06b9299a05d0ae0, State: Running, Role: LEADER
I20260808 10:39:10.123260 22954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261691cb05f84358a06b9299a05d0ae0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44709 } }
I20260808 10:39:10.124049 22947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:10.124117 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:10.109101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57403 (local address 127.30.146.190:44709)
0808 10:39:10.109601 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:10.109618 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:10.109656 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:39:10.110389 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:10.110408 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:10.110790 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:10.111233 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:10.112188 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:10.113350 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:10.114672 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:10.115219 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:10.115371 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:10.118474 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:10.118479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:10.118494 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:10.118523 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:10.120664 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:10.121131 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:10.121138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:10.121162 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:10.121305 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:10.121611 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:10.121615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:10.121617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:10.122098 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:10.122291 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:10.122755 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 10:39:10.123116 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":144,"threads_started":1}
I20260808 10:39:10.124543 22955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 261691cb05f84358a06b9299a05d0ae0. Latest consensus state: current_term: 1 leader_uuid: "261691cb05f84358a06b9299a05d0ae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261691cb05f84358a06b9299a05d0ae0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44709 } } }
I20260808 10:39:10.124511 22954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "261691cb05f84358a06b9299a05d0ae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261691cb05f84358a06b9299a05d0ae0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44709 } } }
I20260808 10:39:10.124809 22954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:10.124743 22955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:10.126863 22959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:10.127743 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:10.130296 22959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:10.136833 22959 catalog_manager.cc:1347] Generated new cluster ID: e5a877b6a4da484aa9c0b1839ed118c4
I20260808 10:39:10.136895 22959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:10.138257 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:10.129836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37361 (local address 127.30.146.190:44709)
0808 10:39:10.129947 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:10.129951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:10.130171 (+ 220us) server_negotiation.cc:407] Connection header received
0808 10:39:10.130311 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:10.130315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:10.130360 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:10.130438 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:10.131112 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:10.131633 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:10.132527 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:10.132900 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:10.133016 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:10.134661 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:10.134664 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:10.134667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:10.134698 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:10.136299 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:10.136858 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:10.136861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:10.136864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:10.136950 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:10.137311 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:10.137315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:10.137317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:10.137557 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:10.137699 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:10.137970 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 10:39:10.138100 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 10:39:10.156829 22959 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:10.157866 22959 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:10.168442 22959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 261691cb05f84358a06b9299a05d0ae0: Generated new TSK 0
I20260808 10:39:10.169333 22959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:10.175819 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 22874
2026-08-08T10:39:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:10 dist-test-slave-9gcw krb5kdc[22981](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:10 dist-test-slave-9gcw krb5kdc[22981](info): set up 2 sockets
Aug 08 10:39:10 dist-test-slave-9gcw krb5kdc[22981](info): commencing operation
krb5kdc: starting...
W20260808 10:39:11.281025 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:11 dist-test-slave-9gcw krb5kdc[22981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:39:11.413945 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:34045
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:38541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:34045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:11.527392 22995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:11.527902 22995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:11.527999 22995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:11.532493 22995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:11.532582 22995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:11.532631 22995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:11.532688 22995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:11.532727 22995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:11.538286 22995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:38541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:34045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:34045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:11.539433 22995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:11.540505 22995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:11.549393 23003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:11.549584 22995 server_base.cc:1034] running on GCE node
W20260808 10:39:11.549417 23000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:11.549398 23001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:11.550427 22995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:11.551599 22995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:11.552783 22995 hybrid_clock.cc:648] HybridClock initialized: now 1786185551552734 us; error 48 us; skew 500 ppm
Aug 08 10:39:11 dist-test-slave-9gcw krb5kdc[22981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185551, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:11.555691 22995 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:39:11.556612 22995 webserver.cc:460] Webserver started at http://127.30.146.190:34059/ using document root <none> and password file <none>
I20260808 10:39:11.557246 22995 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:11.557353 22995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:11.557592 22995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:11.559775 22995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c"
format_stamp: "Formatted at 2026-08-08 10:39:11 on dist-test-slave-9gcw"
server_key: "302fe608bbc86839bb3fdfc2df7e9a07"
server_key_iv: "a923ee25936d965aaeb2b172b9e28935"
server_key_version: "encryptionkey@0"
I20260808 10:39:11.560353 22995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c"
format_stamp: "Formatted at 2026-08-08 10:39:11 on dist-test-slave-9gcw"
server_key: "302fe608bbc86839bb3fdfc2df7e9a07"
server_key_iv: "a923ee25936d965aaeb2b172b9e28935"
server_key_version: "encryptionkey@0"
I20260808 10:39:11.564157 22995 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:39:11.566579 23010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:11.567808 22995 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:39:11.567950 22995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c"
format_stamp: "Formatted at 2026-08-08 10:39:11 on dist-test-slave-9gcw"
server_key: "302fe608bbc86839bb3fdfc2df7e9a07"
server_key_iv: "a923ee25936d965aaeb2b172b9e28935"
server_key_version: "encryptionkey@0"
I20260808 10:39:11.568063 22995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:11.599669 22995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:11.602825 22995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:11.603050 22995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:11.610347 22995 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:34045
I20260808 10:39:11.610365 23066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:34045 every 8 connection(s)
I20260808 10:39:11.611338 22995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:11.614265 23067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:11.619953 23067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c: Bootstrap starting.
I20260808 10:39:11.621028 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 22995
I20260808 10:39:11.621222 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:11.621577 31306 external_mini_cluster.cc:1428] Setting key 1a05cc2291e242139115f5e8f554b02d
I20260808 10:39:11.623265 23067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:11.624070 23067 log.cc:824] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:11.626264 23067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c: No bootstrap required, opened a new log
I20260808 10:39:11.631181 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:11.622792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38469 (local address 127.30.146.190:34045)
0808 10:39:11.623222 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:11.623239 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:11.623277 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:39:11.624012 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:11.624031 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:11.624458 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:11.624858 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:11.625268 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:11.626243 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:11.627190 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:11.627917 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:11.628084 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:11.628913 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:11.628919 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:11.628935 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:11.629572 (+ 637us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:11.629652 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:11.629684 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:11.629787 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:11.629914 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 10:39:11.630137 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":141,"threads_started":1}
I20260808 10:39:11.631007 23067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34045 } }
I20260808 10:39:11.631376 23067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:11.631476 23067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a1cda3db7f04a7596ad42c5eaa1cd9c, State: Initialized, Role: FOLLOWER
I20260808 10:39:11.631963 23067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34045 } }
I20260808 10:39:11.632088 23067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:11.632180 23067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:11.632277 23067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:11.633522 23067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34045 } }
I20260808 10:39:11.633898 23067 leader_election.cc:302] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a1cda3db7f04a7596ad42c5eaa1cd9c; no voters:
I20260808 10:39:11.634300 23067 leader_election.cc:288] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:11.634457 23073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:11.634680 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:11.634846 23073 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [term 1 LEADER]: Becoming Leader. State: Replica: 9a1cda3db7f04a7596ad42c5eaa1cd9c, State: Running, Role: LEADER
I20260808 10:39:11.635167 23073 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34045 } }
I20260808 10:39:11.635684 23067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:11.636373 23073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a1cda3db7f04a7596ad42c5eaa1cd9c. Latest consensus state: current_term: 1 leader_uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34045 } } }
I20260808 10:39:11.636500 23073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:11.636866 23074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1cda3db7f04a7596ad42c5eaa1cd9c" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 34045 } } }
I20260808 10:39:11.636981 23074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:11.636981 23086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:11.640702 23086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:11.642509 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:11.637116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36451 (local address 127.30.146.190:34045)
0808 10:39:11.637357 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:11.637363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:11.637878 (+ 515us) server_negotiation.cc:407] Connection header received
0808 10:39:11.637928 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:11.637931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:11.637981 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:11.638063 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:11.638454 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:11.639195 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:11.640363 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:11.640677 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:11.640776 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:11.641842 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:11.641846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:11.641849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:11.641861 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:11.641890 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:11.641904 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:11.641968 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:11.642320 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 10:39:11.642374 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260808 10:39:11.647668 23086 catalog_manager.cc:1347] Generated new cluster ID: 7d8d22592b7b4b3b99c6e2047bcee240
I20260808 10:39:11.647744 23086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:11.664517 23086 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:11.665639 23086 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:11.678082 23086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a1cda3db7f04a7596ad42c5eaa1cd9c: Generated new TSK 0
I20260808 10:39:11.679031 23086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:11.691802 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 22995
2026-08-08T10:39:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:11 dist-test-slave-9gcw krb5kdc[23102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:11 dist-test-slave-9gcw krb5kdc[23102](info): set up 2 sockets
Aug 08 10:39:11 dist-test-slave-9gcw krb5kdc[23102](info): commencing operation
krb5kdc: starting...
W20260808 10:39:12.816349 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:12 dist-test-slave-9gcw krb5kdc[23102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:39:12.961225 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:35337
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:35993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:35337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:13.086386 23117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:13.086687 23117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:13.086788 23117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:13.091508 23117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:13.091572 23117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:13.091593 23117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:13.091655 23117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:13.091683 23117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:13.097080 23117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:35993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:35337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:35337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:13.098162 23117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:13.099225 23117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:13.108197 23124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:13.108152 23123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:13.108232 23117 server_base.cc:1034] running on GCE node
W20260808 10:39:13.108177 23126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:13.109025 23117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:13.110410 23117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:13.111615 23117 hybrid_clock.cc:648] HybridClock initialized: now 1786185553111585 us; error 44 us; skew 500 ppm
Aug 08 10:39:13 dist-test-slave-9gcw krb5kdc[23102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185553, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:13.114699 23117 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:39:13.115721 23117 webserver.cc:460] Webserver started at http://127.30.146.190:36607/ using document root <none> and password file <none>
I20260808 10:39:13.116267 23117 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:13.116328 23117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:13.116564 23117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:13.118811 23117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "277c96111188401ea3210cb4c6514079"
format_stamp: "Formatted at 2026-08-08 10:39:13 on dist-test-slave-9gcw"
server_key: "fb0bba0b9ec2897e885fb7e3b64c0c60"
server_key_iv: "9291adf74828bfc03d7dfbfb89948cd5"
server_key_version: "encryptionkey@0"
I20260808 10:39:13.119421 23117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "277c96111188401ea3210cb4c6514079"
format_stamp: "Formatted at 2026-08-08 10:39:13 on dist-test-slave-9gcw"
server_key: "fb0bba0b9ec2897e885fb7e3b64c0c60"
server_key_iv: "9291adf74828bfc03d7dfbfb89948cd5"
server_key_version: "encryptionkey@0"
I20260808 10:39:13.123421 23117 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:13.126061 23134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:13.127372 23117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:39:13.127511 23117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "277c96111188401ea3210cb4c6514079"
format_stamp: "Formatted at 2026-08-08 10:39:13 on dist-test-slave-9gcw"
server_key: "fb0bba0b9ec2897e885fb7e3b64c0c60"
server_key_iv: "9291adf74828bfc03d7dfbfb89948cd5"
server_key_version: "encryptionkey@0"
I20260808 10:39:13.127620 23117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:13.133286 23117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:13.136981 23117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:13.137264 23117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:13.145426 23117 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:35337
I20260808 10:39:13.145465 23190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:35337 every 8 connection(s)
I20260808 10:39:13.146468 23117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:13.149430 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 23117
I20260808 10:39:13.149439 23191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:13.149600 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:13.149924 31306 external_mini_cluster.cc:1428] Setting key d1219021b4e8a354a2759dc99c66264a
I20260808 10:39:13.156028 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:13.151406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38449 (local address 127.30.146.190:35337)
0808 10:39:13.151844 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:13.151866 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:13.151911 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:39:13.152992 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:13.153017 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:13.153434 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:13.153807 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:13.154585 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:13.154590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:13.154605 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:13.154617 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:13.154682 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:13.154708 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:13.154851 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:13.154920 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 10:39:13.155078 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
I20260808 10:39:13.157212 23191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079: Bootstrap starting.
I20260808 10:39:13.159124 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:13.160105 23191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:13.161099 23191 log.cc:824] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:13.163372 23191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079: No bootstrap required, opened a new log
I20260808 10:39:13.163550 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:13.161640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59621 (local address 127.30.146.190:35337)
0808 10:39:13.161794 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:13.161799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:13.162018 (+ 219us) server_negotiation.cc:407] Connection header received
0808 10:39:13.162165 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:13.162168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:13.162208 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:13.162292 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:13.163053 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:13.163056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:13.163059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:39:13.163070 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:13.163105 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:39:13.163152 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:13.163247 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:13.163331 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 10:39:13.163372 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:39:13.167194 23191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277c96111188401ea3210cb4c6514079" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35337 } }
I20260808 10:39:13.167423 23191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:39:13.167500 23191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277c96111188401ea3210cb4c6514079, State: Initialized, Role: FOLLOWER
I20260808 10:39:13.167965 23191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277c96111188401ea3210cb4c6514079" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35337 } }
I20260808 10:39:13.168115 23191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:39:13.168210 23191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:39:13.168334 23191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:13.169404 23191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277c96111188401ea3210cb4c6514079" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35337 } }
I20260808 10:39:13.169744 23191 leader_election.cc:302] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 277c96111188401ea3210cb4c6514079; no voters:
I20260808 10:39:13.170073 23191 leader_election.cc:288] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:39:13.170220 23202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:13.170440 23202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [term 1 LEADER]: Becoming Leader. State: Replica: 277c96111188401ea3210cb4c6514079, State: Running, Role: LEADER
I20260808 10:39:13.170747 23202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277c96111188401ea3210cb4c6514079" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35337 } }
I20260808 10:39:13.171414 23191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:13.172384 23203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "277c96111188401ea3210cb4c6514079" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277c96111188401ea3210cb4c6514079" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35337 } } }
I20260808 10:39:13.172713 23204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 277c96111188401ea3210cb4c6514079. Latest consensus state: current_term: 1 leader_uuid: "277c96111188401ea3210cb4c6514079" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277c96111188401ea3210cb4c6514079" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 35337 } } }
I20260808 10:39:13.172892 23204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:13.173044 23203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:13.173964 23212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:13.176537 23212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:13.182749 23212 catalog_manager.cc:1347] Generated new cluster ID: eedc455271a243eeae7f48e86006fd76
I20260808 10:39:13.182832 23212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:13.205030 23212 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:13.205426 23212 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:13.217952 23212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 277c96111188401ea3210cb4c6514079: Generated new TSK 0
I20260808 10:39:13.218875 23212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:39:13.231657 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 23117
2026-08-08T10:39:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1537 ms)
[----------] 18 tests from AuthTokenIssuingTest (27482 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:13 dist-test-slave-9gcw krb5kdc[23225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:13 dist-test-slave-9gcw krb5kdc[23225](info): set up 2 sockets
Aug 08 10:39:13 dist-test-slave-9gcw krb5kdc[23225](info): commencing operation
krb5kdc: starting...
W20260808 10:39:14.348734 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:14 dist-test-slave-9gcw krb5kdc[23225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:39:14.485088 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:44107
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:44107,127.30.146.189:39041,127.30.146.188:44041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:14.603112 23240 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:39:14.603432 23240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:14.603457 23240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:14.603550 23240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:14.607754 23240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:14.607815 23240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:14.607836 23240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:14.607852 23240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:14.607868 23240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:14.613365 23240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:44107,127.30.146.189:39041,127.30.146.188:44041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:44107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:14.614476 23240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:14.615590 23240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:14.625298 23248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:14.625216 23246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:14.625237 23245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:14.625525 23240 server_base.cc:1034] running on GCE node
I20260808 10:39:14.626121 23240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:14.627274 23240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:14.628445 23240 hybrid_clock.cc:648] HybridClock initialized: now 1786185554628423 us; error 36 us; skew 500 ppm
Aug 08 10:39:14 dist-test-slave-9gcw krb5kdc[23225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:14.638494 23240 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:39:14.639541 23240 webserver.cc:460] Webserver started at http://127.30.146.190:41095/ using document root <none> and password file <none>
I20260808 10:39:14.640102 23240 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:14.640164 23240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:14.640350 23240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:14.642537 23240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "cffbb6dba791462988bfe49cce2f12b1"
format_stamp: "Formatted at 2026-08-08 10:39:14 on dist-test-slave-9gcw"
server_key: "f452b8e503f51dd71339e33422c0af23"
server_key_iv: "3a03885812d268fd4728d622e11ecfa1"
server_key_version: "encryptionkey@0"
I20260808 10:39:14.643054 23240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "cffbb6dba791462988bfe49cce2f12b1"
format_stamp: "Formatted at 2026-08-08 10:39:14 on dist-test-slave-9gcw"
server_key: "f452b8e503f51dd71339e33422c0af23"
server_key_iv: "3a03885812d268fd4728d622e11ecfa1"
server_key_version: "encryptionkey@0"
I20260808 10:39:14.646883 23240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:14.649463 23256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:14.650758 23240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:14.650904 23240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "cffbb6dba791462988bfe49cce2f12b1"
format_stamp: "Formatted at 2026-08-08 10:39:14 on dist-test-slave-9gcw"
server_key: "f452b8e503f51dd71339e33422c0af23"
server_key_iv: "3a03885812d268fd4728d622e11ecfa1"
server_key_version: "encryptionkey@0"
I20260808 10:39:14.651021 23240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:14.681456 23240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:14.684950 23240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:14.685194 23240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:14.692593 23240 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:44107
I20260808 10:39:14.692894 23315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:44107 every 8 connection(s)
I20260808 10:39:14.693645 23240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:14.696436 23316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:14.700103 23316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:14.703635 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 23240
I20260808 10:39:14.703774 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:14.704078 31306 external_mini_cluster.cc:1428] Setting key de7892cf29df37fd3913c91e08ea8509
Aug 08 10:39:14 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.146.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.189@KRBTEST.COM" created.
I20260808 10:39:14.717411 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.702362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:44107 (local address 127.0.0.1:46500)
0808 10:39:14.702955 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.703007 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:14.704157 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:14.704864 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:14.704872 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:14.705421 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:14.705828 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:14.705849 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.707540 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.707543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:14.708282 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:14.708291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.709341 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.709347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:14.709452 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:14.710180 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:14.710197 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:14.711786 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:14.714130 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:14.714139 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:14.714147 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:14.714395 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:14.714792 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:14.714799 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:14.714801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:14.714908 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:14.715573 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:14.715579 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:14.715715 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 10:39:14.715902 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 10:39:14.716322 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":108,"spinlock_wait_cycles":30976,"thread_start_us":162,"threads_started":1}
I20260808 10:39:14.717411 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.702362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46500 (local address 127.30.146.190:44107)
0808 10:39:14.702874 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:14.702894 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:14.703113 (+ 219us) server_negotiation.cc:407] Connection header received
0808 10:39:14.704477 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:14.704481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:14.704656 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:14.704806 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:14.705988 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.707397 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.708412 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.709094 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.709330 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:14.711955 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:14.711963 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:14.711979 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:14.712007 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:14.714016 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:14.714560 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:14.714565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:14.714566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:14.714672 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:14.715020 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:14.715023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:14.715025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:14.715452 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:14.715602 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:14.716014 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 10:39:14.716238 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":140,"threads_started":1}
I20260808 10:39:14.719892 23316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:14.720757 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.720405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:39041 (local address 127.0.0.1:51030)
0808 10:39:14.720564 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.720659 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 10:39:14.721050 23261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.146.189:39041: connect: Connection refused (error 111)
I20260808 10:39:14.722299 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.721996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:39041 (local address 127.0.0.1:51036)
0808 10:39:14.722129 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.722198 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 10:39:14.722489 23316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.189:39041: Network error: Client connection negotiation failed: client connection to 127.30.146.189:39041: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab.
Entry for principal kudu/127.30.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab.
I20260808 10:39:14.736086 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.189:39041
--webserver_interface=127.30.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab
--principal=kudu/127.30.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:44107,127.30.146.189:39041,127.30.146.188:44041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:39:14.771771 23316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } attempt: 1
I20260808 10:39:14.772712 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.772397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:39041 (local address 127.0.0.1:51038)
0808 10:39:14.772566 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.772633 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 10:39:14.773501 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.773279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:39041 (local address 127.0.0.1:51050)
0808 10:39:14.773388 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.773443 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 10:39:14.773658 23316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.189:39041: Network error: Client connection negotiation failed: client connection to 127.30.146.189:39041: connect: Connection refused (error 111)
I20260808 10:39:14.841822 23316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } attempt: 2
I20260808 10:39:14.842715 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.842414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:39041 (local address 127.0.0.1:51060)
0808 10:39:14.842569 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.842636 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 10:39:14.843467 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.843235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:39041 (local address 127.0.0.1:51072)
0808 10:39:14.843356 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.843408 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 10:39:14.843597 23316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.189:39041: Network error: Client connection negotiation failed: client connection to 127.30.146.189:39041: connect: Connection refused (error 111)
W20260808 10:39:14.853037 23323 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:39:14.853451 23323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:14.853514 23323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:14.853587 23323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:14.857885 23323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:14.857972 23323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:14.858009 23323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:14.858067 23323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:14.858104 23323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:14.863838 23323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:44107,127.30.146.189:39041,127.30.146.188:44041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab
--principal=kudu/127.30.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.189:39041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:14.865229 23323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:14.866305 23323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:14.875741 23331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:14.875849 23333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:14.875804 23323 server_base.cc:1034] running on GCE node
W20260808 10:39:14.875744 23330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:14.876806 23323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:14.878064 23323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:14.879257 23323 hybrid_clock.cc:648] HybridClock initialized: now 1786185554879243 us; error 33 us; skew 500 ppm
Aug 08 10:39:14 dist-test-slave-9gcw krb5kdc[23225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:14.882213 23323 init.cc:377] Logged in from keytab as kudu/127.30.146.189@KRBTEST.COM (short username kudu)
I20260808 10:39:14.883136 23323 webserver.cc:460] Webserver started at http://127.30.146.189:41477/ using document root <none> and password file <none>
I20260808 10:39:14.883692 23323 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:14.883808 23323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:14.884037 23323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:14.886178 23323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/data/instance:
uuid: "fc70f39e55b74e71909c7424357dc36f"
format_stamp: "Formatted at 2026-08-08 10:39:14 on dist-test-slave-9gcw"
server_key: "5d070d0de02770382b7d35a48af8a8c7"
server_key_iv: "de0718486dd4e3fb901e15a42aa417e3"
server_key_version: "encryptionkey@0"
I20260808 10:39:14.886750 23323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/wal/instance:
uuid: "fc70f39e55b74e71909c7424357dc36f"
format_stamp: "Formatted at 2026-08-08 10:39:14 on dist-test-slave-9gcw"
server_key: "5d070d0de02770382b7d35a48af8a8c7"
server_key_iv: "de0718486dd4e3fb901e15a42aa417e3"
server_key_version: "encryptionkey@0"
I20260808 10:39:14.890461 23323 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:39:14.892982 23340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:14.894227 23323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:14.894368 23323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/wal
uuid: "fc70f39e55b74e71909c7424357dc36f"
format_stamp: "Formatted at 2026-08-08 10:39:14 on dist-test-slave-9gcw"
server_key: "5d070d0de02770382b7d35a48af8a8c7"
server_key_iv: "de0718486dd4e3fb901e15a42aa417e3"
server_key_version: "encryptionkey@0"
I20260808 10:39:14.894480 23323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:14.914579 23323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:14.917835 23323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:14.918051 23323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:14.926304 23323 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.189:39041
I20260808 10:39:14.926608 23396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.189:39041 every 8 connection(s)
I20260808 10:39:14.927490 23323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
I20260808 10:39:14.931308 23397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:14.933411 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 23323
I20260808 10:39:14.933548 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-1/wal/instance
I20260808 10:39:14.933851 31306 external_mini_cluster.cc:1428] Setting key 772d2727ca0d5a1201571f8ea0d282ed
I20260808 10:39:14.934746 23316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } attempt: 3
I20260808 10:39:14.936124 23397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.146.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.188@KRBTEST.COM" created.
Aug 08 10:39:14 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
Aug 08 10:39:14 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:39:14.953074 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.935279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:39041 (local address 127.0.0.1:51080)
0808 10:39:14.935533 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.935546 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:14.935627 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:14.939088 (+ 3461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:14.939092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:14.939332 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:14.939498 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:14.939507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.941643 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.941647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:14.942321 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:14.942331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.942993 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.942997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:14.943177 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:14.943923 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:14.943943 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:14.948114 (+ 4171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:14.950661 (+ 2547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:14.950664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:14.950667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:14.950933 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:14.951384 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:14.951390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:14.951395 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:14.951505 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:14.952329 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:14.952335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:14.952465 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 10:39:14.952664 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 10:39:14.952848 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":47}
I20260808 10:39:14.954341 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.938050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:44107 (local address 127.0.0.1:46516)
0808 10:39:14.938449 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.938509 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:14.938821 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:14.939348 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:14.939355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:14.939671 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:14.940133 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:14.940143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.940923 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.940928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:14.941936 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:14.941947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.943383 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.943392 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:14.943573 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:14.944110 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:14.944132 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:14.949225 (+ 5093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:14.951479 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:14.951491 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:14.951497 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:14.951847 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:14.952213 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:14.952217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:14.952219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:14.952319 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:14.952847 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:14.952854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:14.952970 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 10:39:14.953116 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 10:39:14.953316 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":162,"threads_started":1}
I20260808 10:39:14.954360 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.936763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51080 (local address 127.30.146.189:39041)
0808 10:39:14.937294 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:14.937316 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:14.937364 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:39:14.938497 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:14.938522 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:14.938863 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:14.939680 (+ 817us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:14.940019 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.941497 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.942468 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.942864 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.943048 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:14.948290 (+ 5242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:14.948296 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:14.948306 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:14.948329 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:14.950499 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:14.951070 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:14.951077 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:14.951085 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:14.951246 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:14.951614 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:14.951620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:14.951625 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:14.952199 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:14.952336 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:14.952829 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 10:39:14.953206 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":313,"spinlock_wait_cycles":896,"thread_start_us":123,"threads_started":1}
I20260808 10:39:14.955416 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.938061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46516 (local address 127.30.146.190:44107)
0808 10:39:14.938189 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:14.938196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:14.938609 (+ 413us) server_negotiation.cc:407] Connection header received
0808 10:39:14.939027 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:14.939030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:14.939096 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:14.939181 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:14.940275 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.940809 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.942078 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.943209 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.943333 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:14.949403 (+ 6070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:14.949410 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:14.949413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:14.949442 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:14.951363 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:14.951980 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:14.951986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:14.951990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:14.952090 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:14.952414 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:14.952417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:14.952419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:14.952703 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:14.954983 (+ 2280us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:14.955100 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 10:39:14.955230 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":49}
I20260808 10:39:14.956555 23397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:14.957187 23316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:14.958034 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.957692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42050)
0808 10:39:14.957842 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.957943 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 10:39:14.958871 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.958591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42054)
0808 10:39:14.958752 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.958810 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260808 10:39:14.959038 23316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Aug 08 10:39:14 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
I20260808 10:39:14.966493 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.957338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:39041 (local address 127.0.0.1:51090)
0808 10:39:14.957473 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.957509 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:14.957596 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:14.957975 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:14.957978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:14.958215 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:14.958337 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:14.958346 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.959281 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.959284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:14.960137 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:14.960148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.960714 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.960716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:14.960808 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:14.961419 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:14.961433 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:14.962858 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:14.964797 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:14.964800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:14.964801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:14.965005 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:14.965481 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:14.965484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:14.965485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:14.965629 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:14.965996 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:14.965999 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:14.966140 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 10:39:14.966225 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:39:14.966369 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 10:39:14.966764 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.957490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51090 (local address 127.30.146.189:39041)
0808 10:39:14.957731 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:14.957736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:14.957748 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:39:14.957786 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:14.957788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:14.957831 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:14.957902 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:14.958486 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.959186 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.960294 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:14.960563 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:14.960646 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:14.962991 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:14.962994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:14.962996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:14.963017 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:14.964608 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:14.965206 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:14.965213 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:14.965216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:14.965303 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:14.965722 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:14.965724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:14.965725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:14.965905 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:14.965984 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:14.966296 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 10:39:14.966450 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":178}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab.
Entry for principal kudu/127.30.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab.
I20260808 10:39:14.967711 23397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:14.968477 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.968205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42058)
0808 10:39:14.968328 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.968426 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 10:39:14.968497 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.188:44041
--webserver_interface=127.30.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab
--principal=kudu/127.30.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:37667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:44107,127.30.146.189:39041,127.30.146.188:44041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:14.969290 23343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
I20260808 10:39:14.970232 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.970034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42062)
0808 10:39:14.970142 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.970185 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 10:39:14.970487 23397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
I20260808 10:39:14.990231 23316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } attempt: 1
I20260808 10:39:14.991233 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.990876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42068)
0808 10:39:14.991047 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.991142 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 10:39:14.992115 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:14.991849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42082)
0808 10:39:14.991988 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:14.992050 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 10:39:14.992319 23316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
I20260808 10:39:15.019819 23397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } attempt: 1
I20260808 10:39:15.020910 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.020500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42090)
0808 10:39:15.020750 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.020827 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260808 10:39:15.021883 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.021551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42106)
0808 10:39:15.021741 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.021813 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260808 10:39:15.022032 23397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
I20260808 10:39:15.067507 23316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } attempt: 2
I20260808 10:39:15.068473 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.068152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42112)
0808 10:39:15.068318 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.068385 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 10:39:15.069293 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.069011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42124)
0808 10:39:15.069139 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.069232 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 10:39:15.069437 23316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
W20260808 10:39:15.089825 23404 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:39:15.090148 23404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:15.090222 23404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:15.090304 23404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 10:39:15.090225 23397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } attempt: 2
I20260808 10:39:15.091178 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.090890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42130)
0808 10:39:15.091041 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.091103 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 10:39:15.091893 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.091669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42142)
0808 10:39:15.091781 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.091835 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 10:39:15.092027 23397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.146.188:44041: connect: Connection refused (error 111)
W20260808 10:39:15.094915 23404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:15.094991 23404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:15.095044 23404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:15.095077 23404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:15.095103 23404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:15.100607 23404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:37667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:44107,127.30.146.189:39041,127.30.146.188:44041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab
--principal=kudu/127.30.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.188:44041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:15.101750 23404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:15.102777 23404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:15.111579 23412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:15.111579 23411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:15.111711 23414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:15.111869 23404 server_base.cc:1034] running on GCE node
I20260808 10:39:15.112560 23404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:15.113700 23404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:15.114889 23404 hybrid_clock.cc:648] HybridClock initialized: now 1786185555114874 us; error 29 us; skew 500 ppm
Aug 08 10:39:15 dist-test-slave-9gcw krb5kdc[23225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185555, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:15.117810 23404 init.cc:377] Logged in from keytab as kudu/127.30.146.188@KRBTEST.COM (short username kudu)
I20260808 10:39:15.118818 23404 webserver.cc:460] Webserver started at http://127.30.146.188:34733/ using document root <none> and password file <none>
I20260808 10:39:15.119364 23404 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:15.119424 23404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:15.119593 23404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:15.121801 23404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/data/instance:
uuid: "7d054681ddd34fc58a0e399f58d0513b"
format_stamp: "Formatted at 2026-08-08 10:39:15 on dist-test-slave-9gcw"
server_key: "ca209ab6b86b328c90ac19ba25791dd4"
server_key_iv: "b8a777c61996b473f2ac22415715e8ff"
server_key_version: "encryptionkey@0"
I20260808 10:39:15.122345 23404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/wal/instance:
uuid: "7d054681ddd34fc58a0e399f58d0513b"
format_stamp: "Formatted at 2026-08-08 10:39:15 on dist-test-slave-9gcw"
server_key: "ca209ab6b86b328c90ac19ba25791dd4"
server_key_iv: "b8a777c61996b473f2ac22415715e8ff"
server_key_version: "encryptionkey@0"
I20260808 10:39:15.126325 23404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 10:39:15.128836 23422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:15.130167 23404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:39:15.130311 23404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/wal
uuid: "7d054681ddd34fc58a0e399f58d0513b"
format_stamp: "Formatted at 2026-08-08 10:39:15 on dist-test-slave-9gcw"
server_key: "ca209ab6b86b328c90ac19ba25791dd4"
server_key_iv: "b8a777c61996b473f2ac22415715e8ff"
server_key_version: "encryptionkey@0"
I20260808 10:39:15.130403 23404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:15.144927 23404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:15.147881 23404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:15.148038 23404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:15.155649 23404 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.188:44041
I20260808 10:39:15.155947 23480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.188:44041 every 8 connection(s)
I20260808 10:39:15.156764 23404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
I20260808 10:39:15.159704 23481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:15.163736 23481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:15.166416 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 23404
I20260808 10:39:15.166544 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185256684052-31306-0/minicluster-data/master-2/wal/instance
I20260808 10:39:15.166831 31306 external_mini_cluster.cc:1428] Setting key e00ab09c924118a6ba8633900f5337fe
I20260808 10:39:15.168608 23316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } attempt: 3
Aug 08 10:39:15 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
Aug 08 10:39:15 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185555, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
Aug 08 10:39:15 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
I20260808 10:39:15.179662 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.168981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42160)
0808 10:39:15.169075 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.169091 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:15.169193 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:15.169629 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:15.169631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:15.169813 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:15.169909 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:15.169917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.171187 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.171191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:15.171966 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:15.171979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.172768 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.172772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:15.172847 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.173573 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:15.173592 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:15.174888 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:15.177525 (+ 2637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:15.177529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:15.177531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:15.177841 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:15.178245 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:15.178248 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:15.178250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:15.178362 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:15.179074 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:15.179078 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:15.179185 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 10:39:15.179302 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 10:39:15.179452 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 10:39:15.179811 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.165737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46518 (local address 127.30.146.190:44107)
0808 10:39:15.165926 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.165931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.167032 (+ 1101us) server_negotiation.cc:407] Connection header received
0808 10:39:15.168242 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.168245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.168302 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.168400 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.170094 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.170665 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.172082 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.172440 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.172546 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.175797 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.175802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.175809 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.175831 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.177716 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.178209 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.178212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.178215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.178305 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.178693 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.178697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.178698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.178960 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.179034 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.179538 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.179672 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:39:15.180754 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.166170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:44107 (local address 127.0.0.1:46518)
0808 10:39:15.166849 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.166920 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:15.168020 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:15.168451 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:15.168457 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:15.169702 (+ 1245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:15.169935 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:15.169950 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.170794 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.170798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:15.171921 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:15.171933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.172602 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.172606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:15.172776 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.173951 (+ 1175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:15.173977 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:15.175629 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:15.177825 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:15.177835 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:15.177845 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:15.178104 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:15.178402 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:15.178405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:15.178407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:15.178553 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:15.179115 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:15.179121 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:15.179257 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 10:39:15.179996 (+ 739us) client_negotiation.cc:240] Negotiation successful
0808 10:39:15.180186 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"spinlock_wait_cycles":896,"thread_start_us":138,"threads_started":1}
I20260808 10:39:15.180754 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.169069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42160 (local address 127.30.146.188:44041)
0808 10:39:15.169405 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.169415 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.169427 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:39:15.169456 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.169458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.169496 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.169581 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.170048 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.171056 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.172082 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.172641 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.172796 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.175063 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.175071 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.175086 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.175118 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.177398 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.177965 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.177970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.177972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.178111 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.178450 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.178456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.178459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.178942 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.179111 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.179402 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.179692 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":134,"server-negotiator.queue_time_us":265,"thread_start_us":148,"threads_started":1}
I20260808 10:39:15.182205 23481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:15.182273 23485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.167478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42156 (local address 127.30.146.188:44041)
0808 10:39:15.167871 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.167880 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.167915 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:39:15.168086 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.168100 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.168273 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.168463 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.169455 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.171056 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.173028 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.173408 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.173497 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.176829 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.176835 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.176839 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.176868 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.179609 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.180471 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.180474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.180476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.180563 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.180845 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.180851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.180854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.181396 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.181595 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.181955 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.182079 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":305,"thread_start_us":164,"threads_started":1}
I20260808 10:39:15.183333 23397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } attempt: 3
I20260808 10:39:15.186112 23316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1: Bootstrap starting.
Aug 08 10:39:15 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185555, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
I20260808 10:39:15.189103 23316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:15.190037 23316 log.cc:824] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:15.191744 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.182853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51094 (local address 127.30.146.189:39041)
0808 10:39:15.182993 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.182997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.183276 (+ 279us) server_negotiation.cc:407] Connection header received
0808 10:39:15.183394 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.183397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.183451 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.183536 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.184120 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.184700 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.185513 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.185876 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.186005 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.188143 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.188146 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.188148 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.188172 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.189705 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.190215 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.190220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.190222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.190320 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.190777 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.190782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.190783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.191032 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.191148 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.191444 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.191588 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:39:15.192389 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.182715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:39041 (local address 127.0.0.1:51094)
0808 10:39:15.183179 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.183198 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:15.183295 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:15.183667 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:15.183670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:15.183842 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:15.183981 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:15.183989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.184823 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.184825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:15.185394 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:15.185402 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.185995 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.185998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:15.186069 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.186503 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:15.186514 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:15.188004 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:15.189824 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:15.189828 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:15.189830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:15.190107 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:15.190425 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:15.190430 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:15.190433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:15.190541 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:15.191138 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:15.191141 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:15.191243 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 10:39:15.192097 (+ 854us) client_negotiation.cc:240] Negotiation successful
0808 10:39:15.192225 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388}
I20260808 10:39:15.193414 23481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:15.193634 23316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1: No bootstrap required, opened a new log
Aug 08 10:39:15 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
Aug 08 10:39:15 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:39:15.199117 23316 raft_consensus.cc:348] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } }
I20260808 10:39:15.199378 23316 raft_consensus.cc:729] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffbb6dba791462988bfe49cce2f12b1, State: Initialized, Role: FOLLOWER
I20260808 10:39:15.199775 23316 consensus_queue.cc:260] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } }
I20260808 10:39:15.200358 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.183753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42170)
0808 10:39:15.190572 (+ 6819us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.190590 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:15.190678 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:15.191380 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:15.191385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:15.191581 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:15.191668 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:15.191675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.192878 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.192882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:15.193606 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:15.193616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.194374 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.194378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:15.194464 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.194998 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:15.195009 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:15.196495 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:15.198634 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:15.198639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:15.198644 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:15.198892 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:15.199211 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:15.199214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:15.199216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:15.199297 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:15.199754 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:15.199758 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:15.199870 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 10:39:15.200012 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 10:39:15.200174 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6732}
I20260808 10:39:15.200345 23488 sys_catalog.cc:455] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } } }
I20260808 10:39:15.200757 23488 sys_catalog.cc:458] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:15.201429 23316 sys_catalog.cc:564] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:15.201916 23485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.183991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.30.146.188:44041)
0808 10:39:15.184311 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.184315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.191175 (+ 6860us) server_negotiation.cc:407] Connection header received
0808 10:39:15.191209 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.191211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.191249 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.191780 (+ 531us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.191887 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.192606 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.193753 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.194222 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.194377 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.196648 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.196652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.196654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.196687 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.198523 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.199015 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.199018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.199020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.199112 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.199407 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.199409 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.199411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.199622 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.199753 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.201094 (+ 1341us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.201744 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":492,"server-negotiator.queue_time_us":239}
Aug 08 10:39:15 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185555, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
I20260808 10:39:15.205049 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.190820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46526 (local address 127.30.146.190:44107)
0808 10:39:15.190946 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.190950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.190967 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:39:15.191033 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.191036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.191078 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.192471 (+ 1393us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.193489 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.194191 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.195536 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.195908 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.196021 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.199323 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.199329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.199335 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.199365 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.202741 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.203321 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.203325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.203327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.203424 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.203788 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.203792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.203795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.204043 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.204263 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.204760 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.204880 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:39:15.205487 23397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f: Bootstrap starting.
I20260808 10:39:15.207093 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.193908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:44041 (local address 127.0.0.1:42184)
0808 10:39:15.197277 (+ 3369us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:15.197296 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:15.197408 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:15.198040 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:15.198046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:15.198230 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:15.198398 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:15.198408 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.199607 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.199611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:15.200656 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:15.200669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.201400 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.201402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:15.201452 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.202108 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:15.202121 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:15.203407 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:15.205641 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:15.205644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:15.205646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:15.205841 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:15.206155 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:15.206157 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:15.206158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:15.206232 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:15.206719 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:15.206721 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:15.206810 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 10:39:15.206899 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 10:39:15.206991 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3273}
I20260808 10:39:15.207278 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.194361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42184 (local address 127.30.146.188:44041)
0808 10:39:15.195615 (+ 1254us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.195619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.197383 (+ 1764us) server_negotiation.cc:407] Connection header received
0808 10:39:15.197782 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.197787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.197844 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.197931 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.198523 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.199443 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.200785 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.201207 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.201306 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.203542 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.203545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.203548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.203569 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.205486 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.205947 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.205951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.205953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.206050 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.206325 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.206329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.206330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.206559 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.206684 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.206988 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.207113 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1159}
I20260808 10:39:15.208884 23397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:15.209924 23397 log.cc:824] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:15.210140 23481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b: Bootstrap starting.
I20260808 10:39:15.212754 23481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:15.213665 23481 log.cc:824] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b: Log is configured to *not* fsync() on all Append() calls
W20260808 10:39:15.214758 23500 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:15.214895 23500 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 10:39:15 dist-test-slave-9gcw krb5kdc[23225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
I20260808 10:39:15.215754 23481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b: No bootstrap required, opened a new log
I20260808 10:39:15.218690 23397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f: No bootstrap required, opened a new log
I20260808 10:39:15.219533 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.208624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.30.146.189:39041)
0808 10:39:15.208757 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.208768 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.208782 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:39:15.208943 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.208946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.208993 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.209080 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.209955 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.210638 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.211807 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.212106 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.212175 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.215573 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.215577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.215579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.215604 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.217461 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.217974 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.217978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.217980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.218070 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.218395 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.218398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.218400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.218625 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.218806 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.219235 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.219358 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:39:15.215397 23240 master_runner.cc:186] Error getting master registration for 127.30.146.189:39041: OK, instance_id { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" instance_seqno: 1786185554923959 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:39:15.219739 23240 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:39:15.220397 23481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } }
I20260808 10:39:15.220664 23481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d054681ddd34fc58a0e399f58d0513b, State: Initialized, Role: FOLLOWER
I20260808 10:39:15.221026 23481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } }
I20260808 10:39:15.222456 23481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:15.222819 23397 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } }
I20260808 10:39:15.223057 23397 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc70f39e55b74e71909c7424357dc36f, State: Initialized, Role: FOLLOWER
I20260808 10:39:15.223389 23397 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } }
I20260808 10:39:15.223927 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:15.223914 23509 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } } }
I20260808 10:39:15.224069 23509 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:15.224759 23397 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:15.225471 23503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } } }
I20260808 10:39:15.225649 23503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:15.225795 23290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cffbb6dba791462988bfe49cce2f12b1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46526
I20260808 10:39:15.226126 23290 raft_consensus.cc:482] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 10:39:15.226258 23290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:15.227581 23290 raft_consensus.cc:504] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } }
I20260808 10:39:15.228621 23290 leader_election.cc:288] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [CANDIDATE]: Term 1 election: Requested vote from peers fc70f39e55b74e71909c7424357dc36f (127.30.146.189:39041), 7d054681ddd34fc58a0e399f58d0513b (127.30.146.188:44041)
I20260808 10:39:15.229419 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cffbb6dba791462988bfe49cce2f12b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fc70f39e55b74e71909c7424357dc36f"
I20260808 10:39:15.229626 23368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:15.230962 23368 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cffbb6dba791462988bfe49cce2f12b1 in term 1.
I20260808 10:39:15.231062 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cffbb6dba791462988bfe49cce2f12b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7d054681ddd34fc58a0e399f58d0513b"
I20260808 10:39:15.231312 23454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [term 0 FOLLOWER]: Advancing to term 1
W20260808 10:39:15.231520 23515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:15.231599 23515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:15.232028 23261 leader_election.cc:302] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cffbb6dba791462988bfe49cce2f12b1, fc70f39e55b74e71909c7424357dc36f; no voters:
I20260808 10:39:15.232321 23488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:15.232573 23488 raft_consensus.cc:686] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [term 1 LEADER]: Becoming Leader. State: Replica: cffbb6dba791462988bfe49cce2f12b1, State: Running, Role: LEADER
I20260808 10:39:15.232795 23454 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cffbb6dba791462988bfe49cce2f12b1 in term 1.
I20260808 10:39:15.232915 23488 consensus_queue.cc:237] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } }
I20260808 10:39:15.235235 23488 sys_catalog.cc:455] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cffbb6dba791462988bfe49cce2f12b1. Latest consensus state: current_term: 1 leader_uuid: "cffbb6dba791462988bfe49cce2f12b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } } }
I20260808 10:39:15.235388 23488 sys_catalog.cc:458] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [sys.catalog]: This master's current role is: LEADER
W20260808 10:39:15.239674 23528 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:15.239763 23528 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:15.240144 23404 master_runner.cc:263] No leader master found from master 7d054681ddd34fc58a0e399f58d0513b
I20260808 10:39:15.240216 23404 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:39:15.240577 23538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:15.240646 23323 master_runner.cc:263] No leader master found from master fc70f39e55b74e71909c7424357dc36f
I20260808 10:39:15.240723 23323 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:39:15.242089 23538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:39:15.247545 23368 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [term 1 FOLLOWER]: Refusing update from remote peer cffbb6dba791462988bfe49cce2f12b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:15.247612 23454 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [term 1 FOLLOWER]: Refusing update from remote peer cffbb6dba791462988bfe49cce2f12b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:15.248302 23488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:15.248726 23488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:15.252322 23509 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cffbb6dba791462988bfe49cce2f12b1. Latest consensus state: current_term: 1 leader_uuid: "cffbb6dba791462988bfe49cce2f12b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } } }
I20260808 10:39:15.252451 23509 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:15.252579 23503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cffbb6dba791462988bfe49cce2f12b1. Latest consensus state: current_term: 1 leader_uuid: "cffbb6dba791462988bfe49cce2f12b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } } }
I20260808 10:39:15.252692 23503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:15.259212 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.235428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46534 (local address 127.30.146.190:44107)
0808 10:39:15.235559 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.235563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.249458 (+ 13895us) server_negotiation.cc:407] Connection header received
0808 10:39:15.249673 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.249676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.249727 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.249785 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.250885 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.251643 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.252832 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.253203 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.253309 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.255291 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.255295 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.255298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.255348 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.257128 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.257725 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.257731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.257737 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.257838 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.258160 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.258163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.258165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.258408 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.258517 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.258893 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.259031 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:39:15.259184 23540 mvcc.cc:205] Tried to move back new op lower bound from 7316216034286780416 to 7316216034239549440. Current Snapshot: MvccSnapshot[applied={T|T < 7316216034286780416}]
I20260808 10:39:15.259696 23488 sys_catalog.cc:455] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cffbb6dba791462988bfe49cce2f12b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } } }
I20260808 10:39:15.259857 23488 sys_catalog.cc:458] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:15.261178 23503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cffbb6dba791462988bfe49cce2f12b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } } }
I20260808 10:39:15.261343 23503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:15.266359 23544 mvcc.cc:205] Tried to move back new op lower bound from 7316216034286780416 to 7316216034239549440. Current Snapshot: MvccSnapshot[applied={T|T < 7316216034286780416}]
I20260808 10:39:15.273635 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.253434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42200 (local address 127.30.146.188:44041)
0808 10:39:15.253562 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.253578 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.260830 (+ 7252us) server_negotiation.cc:407] Connection header received
0808 10:39:15.261025 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.261028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.261084 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.261138 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.262334 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.263089 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.264229 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.264549 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.264621 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.268773 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.268777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.268780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.268808 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.270601 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.271117 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.271121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.271123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.271212 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.271553 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.271557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.271558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.271789 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.271934 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.273340 (+ 1406us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.273469 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:39:15.273633 23538 catalog_manager.cc:1347] Generated new cluster ID: fa098b5b57284ae4bc5512e112455f0c
I20260808 10:39:15.273705 23538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:15.277707 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:15.245780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.30.146.189:39041)
0808 10:39:15.245902 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.245907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.257797 (+ 11890us) server_negotiation.cc:407] Connection header received
0808 10:39:15.262560 (+ 4763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.262564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.262620 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.262672 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.263597 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.264325 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.265446 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:15.265746 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:15.265838 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:15.274175 (+ 8337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:15.274179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:15.274183 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:15.274214 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:15.275894 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.276404 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.276407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.276409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.276495 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:15.276753 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:15.276755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:15.276757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:15.276945 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:15.277040 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:15.277432 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 10:39:15.277559 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:39:15.289009 23551 sys_catalog.cc:455] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cffbb6dba791462988bfe49cce2f12b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } } }
I20260808 10:39:15.289191 23551 sys_catalog.cc:458] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:15.290496 23509 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cffbb6dba791462988bfe49cce2f12b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffbb6dba791462988bfe49cce2f12b1" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 44107 } } peers { permanent_uuid: "fc70f39e55b74e71909c7424357dc36f" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 39041 } } peers { permanent_uuid: "7d054681ddd34fc58a0e399f58d0513b" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 44041 } } }
I20260808 10:39:15.290628 23509 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:15.291339 23543 mvcc.cc:205] Tried to move back new op lower bound from 7316216034286780416 to 7316216034239549440. Current Snapshot: MvccSnapshot[applied={T|T < 7316216034286780416}]
I20260808 10:39:15.303746 23538 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:15.307548 23538 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:15.337321 23538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cffbb6dba791462988bfe49cce2f12b1: Generated new TSK 0
I20260808 10:39:15.338289 23538 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:39:15.370355 23318 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.367068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46538 (local address 127.30.146.190:44107)
0808 10:39:15.367200 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.367206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.369234 (+ 2028us) server_negotiation.cc:407] Connection header received
0808 10:39:15.369441 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.369445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.369496 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.369551 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.370214 (+ 663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46538: BlockingRecv error: recv got EOF from 127.0.0.1:46538 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 10:39:15.371799 23486 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.367543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42214 (local address 127.30.146.188:44041)
0808 10:39:15.367670 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.367675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.370907 (+ 3232us) server_negotiation.cc:407] Connection header received
0808 10:39:15.371075 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.371079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.371128 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.371180 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.371679 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42214: BlockingRecv error: recv got EOF from 127.0.0.1:42214 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 10:39:15.373243 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.368000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.30.146.189:39041)
0808 10:39:15.368117 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.368122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.372286 (+ 4164us) server_negotiation.cc:407] Connection header received
0808 10:39:15.372455 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.372459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.372509 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.372560 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.373054 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51126: BlockingRecv error: recv got EOF from 127.0.0.1:51126 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:39:15.377808 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.376299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51136 (local address 127.30.146.189:39041)
0808 10:39:15.376401 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.376405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.376838 (+ 433us) server_negotiation.cc:407] Connection header received
0808 10:39:15.377043 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.377046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.377134 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.377210 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.377712 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51136: BlockingRecv error: recv got EOF from 127.0.0.1:51136 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 10:39:15.383370 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.381908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51140 (local address 127.30.146.189:39041)
0808 10:39:15.382015 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.382018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.382489 (+ 471us) server_negotiation.cc:407] Connection header received
0808 10:39:15.382676 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.382680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.382727 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.382776 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.383251 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51140: BlockingRecv error: recv got EOF from 127.0.0.1:51140 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:39:15.389248 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.387585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51154 (local address 127.30.146.189:39041)
0808 10:39:15.387692 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.387695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.388167 (+ 472us) server_negotiation.cc:407] Connection header received
0808 10:39:15.388346 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.388349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.388397 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.388449 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.389068 (+ 619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51154: BlockingRecv error: recv got EOF from 127.0.0.1:51154 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 10:39:15.396840 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.395361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.30.146.189:39041)
0808 10:39:15.395491 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.395495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.395975 (+ 480us) server_negotiation.cc:407] Connection header received
0808 10:39:15.396151 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.396154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.396203 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.396253 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.396739 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51170: BlockingRecv error: recv got EOF from 127.0.0.1:51170 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 10:39:15.408849 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.407252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.30.146.189:39041)
0808 10:39:15.407366 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.407370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.407899 (+ 529us) server_negotiation.cc:407] Connection header received
0808 10:39:15.408099 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.408103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.408155 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.408209 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.408736 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51182: BlockingRecv error: recv got EOF from 127.0.0.1:51182 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 10:39:15.431556 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.429643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.30.146.189:39041)
0808 10:39:15.429790 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.429795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.430406 (+ 611us) server_negotiation.cc:407] Connection header received
0808 10:39:15.430637 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.430643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.430729 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.430789 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.431425 (+ 636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51194: BlockingRecv error: recv got EOF from 127.0.0.1:51194 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 10:39:15.468302 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.466781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51208 (local address 127.30.146.189:39041)
0808 10:39:15.466921 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.466927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.467273 (+ 346us) server_negotiation.cc:407] Connection header received
0808 10:39:15.467423 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.467427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.467486 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.467576 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.468171 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51208: BlockingRecv error: recv got EOF from 127.0.0.1:51208 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 10:39:15.535866 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.534597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.30.146.189:39041)
0808 10:39:15.534804 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.534809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.534962 (+ 153us) server_negotiation.cc:407] Connection header received
0808 10:39:15.535062 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.535065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.535130 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.535217 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.535741 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51224: BlockingRecv error: recv got EOF from 127.0.0.1:51224 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
W20260808 10:39:15.667538 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.666174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.30.146.189:39041)
0808 10:39:15.666347 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.666354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.666532 (+ 178us) server_negotiation.cc:407] Connection header received
0808 10:39:15.666656 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.666659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.666714 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.666800 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.667354 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51230: BlockingRecv error: recv got EOF from 127.0.0.1:51230 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 10:39:15.927085 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:15.925963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.30.146.189:39041)
0808 10:39:15.926154 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:15.926160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:15.926182 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:39:15.926347 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:15.926350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:15.926405 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:15.926490 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:15.926927 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51232: BlockingRecv error: recv got EOF from 127.0.0.1:51232 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 10:39:16.232910 23515 catalog_manager.cc:1259] Loaded cluster ID: fa098b5b57284ae4bc5512e112455f0c
I20260808 10:39:16.232992 23515 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b: loading cluster ID for follower catalog manager: success
I20260808 10:39:16.235157 23515 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b: acquiring CA information for follower catalog manager: success
I20260808 10:39:16.236334 23515 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d054681ddd34fc58a0e399f58d0513b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:39:16.241001 23528 catalog_manager.cc:1259] Loaded cluster ID: fa098b5b57284ae4bc5512e112455f0c
I20260808 10:39:16.241117 23528 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f: loading cluster ID for follower catalog manager: success
I20260808 10:39:16.242859 23528 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f: acquiring CA information for follower catalog manager: success
I20260808 10:39:16.244020 23528 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fc70f39e55b74e71909c7424357dc36f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:39:16.445545 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:16.441365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51248 (local address 127.30.146.189:39041)
0808 10:39:16.441681 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:16.441685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:16.441799 (+ 114us) server_negotiation.cc:407] Connection header received
0808 10:39:16.441919 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:16.441922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:16.442060 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:16.442185 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:16.442409 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:16.443103 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:16.443927 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:16.444198 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:16.444291 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:16.444549 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:16.445072 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:16.445255 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:16.445289 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 10:39:16.445341 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":89,"threads_started":1}
I20260808 10:39:16.452407 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:16.448705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42224 (local address 127.30.146.188:44041)
0808 10:39:16.448984 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:16.448988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:16.449002 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:39:16.449082 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:16.449084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:16.449278 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:16.449382 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:16.449554 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:16.450207 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:16.450962 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:16.451213 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:16.451302 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:16.451553 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:16.451974 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:16.452083 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:16.452138 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 10:39:16.452187 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":105,"threads_started":1}
I20260808 10:39:16.459987 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:16.455766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.30.146.188:44041)
0808 10:39:16.455914 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:16.455918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:16.456287 (+ 369us) server_negotiation.cc:407] Connection header received
0808 10:39:16.456391 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:16.456397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:16.456460 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:16.456535 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:16.456787 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:16.457527 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:16.458659 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:16.458995 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:16.459053 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:16.459512 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:16.459626 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:16.459711 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:16.459798 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 10:39:16.459849 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:39:16.460088 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:16.455534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.30.146.189:39041)
0808 10:39:16.455664 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:16.455669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:16.456417 (+ 748us) server_negotiation.cc:407] Connection header received
0808 10:39:16.456606 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:16.456611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:16.456666 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:16.456705 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:16.457091 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:16.457878 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:16.458731 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:16.459039 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:16.459146 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:16.459583 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:16.459671 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:16.459741 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:16.459898 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 10:39:16.459953 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:39:16.460597 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:16.455564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46544 (local address 127.30.146.190:44107)
0808 10:39:16.455980 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:16.455984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:16.456145 (+ 161us) server_negotiation.cc:407] Connection header received
0808 10:39:16.456264 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:16.456267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:16.456450 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:16.456577 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:16.456937 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:16.457830 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:16.458924 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:16.459262 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:16.459374 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:16.459700 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:16.460042 (+ 342us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:16.460134 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:16.460361 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 10:39:16.460424 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":71,"threads_started":1}
I20260808 10:39:16.464088 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 23240
I20260808 10:39:16.472922 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 23323
I20260808 10:39:16.481518 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 23404
2026-08-08T10:39:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3252 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:39:16 dist-test-slave-9gcw krb5kdc[23643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:39:16 dist-test-slave-9gcw krb5kdc[23643](info): set up 2 sockets
Aug 08 10:39:16 dist-test-slave-9gcw krb5kdc[23643](info): commencing operation
krb5kdc: starting...
W20260808 10:39:17.576884 31306 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/test-user.keytab.
Aug 08 10:39:17 dist-test-slave-9gcw krb5kdc[23643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:39:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:39:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
Entry for principal kudu/127.30.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab.
I20260808 10:39:17.713014 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:40131
--webserver_interface=127.30.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:36623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:40131,127.30.146.189:46769,127.30.146.188:40727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:39:17.830097 23656 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:39:17.830504 23656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:17.830600 23656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:17.830717 23656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:17.834918 23656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:17.835021 23656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:17.835063 23656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:17.835107 23656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:17.835150 23656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:17.840864 23656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:36623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:40131,127.30.146.189:46769,127.30.146.188:40727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.190.keytab
--principal=kudu/127.30.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.190:40131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:17.842087 23656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:17.843186 23656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:17.852280 23662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:17.852304 23661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:17.852375 23664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:17.852471 23656 server_base.cc:1034] running on GCE node
I20260808 10:39:17.853418 23656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:17.854622 23656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:17.855829 23656 hybrid_clock.cc:648] HybridClock initialized: now 1786185557855817 us; error 31 us; skew 500 ppm
Aug 08 10:39:17 dist-test-slave-9gcw krb5kdc[23643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185557, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:17.859189 23656 init.cc:377] Logged in from keytab as kudu/127.30.146.190@KRBTEST.COM (short username kudu)
I20260808 10:39:17.860134 23656 webserver.cc:460] Webserver started at http://127.30.146.190:42075/ using document root <none> and password file <none>
I20260808 10:39:17.860723 23656 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:17.860795 23656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:17.861028 23656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:17.863206 23656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/data/instance:
uuid: "04f3b7229a524598b81f313eb2bd5ec0"
format_stamp: "Formatted at 2026-08-08 10:39:17 on dist-test-slave-9gcw"
server_key: "2d4c9a6decc3c3b74c2a78db4e02385a"
server_key_iv: "7fa72116d028967f34320f5287acddbe"
server_key_version: "encryptionkey@0"
I20260808 10:39:17.863794 23656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/wal/instance:
uuid: "04f3b7229a524598b81f313eb2bd5ec0"
format_stamp: "Formatted at 2026-08-08 10:39:17 on dist-test-slave-9gcw"
server_key: "2d4c9a6decc3c3b74c2a78db4e02385a"
server_key_iv: "7fa72116d028967f34320f5287acddbe"
server_key_version: "encryptionkey@0"
I20260808 10:39:17.867681 23656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:39:17.870538 23671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:17.871821 23656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:39:17.871966 23656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/wal
uuid: "04f3b7229a524598b81f313eb2bd5ec0"
format_stamp: "Formatted at 2026-08-08 10:39:17 on dist-test-slave-9gcw"
server_key: "2d4c9a6decc3c3b74c2a78db4e02385a"
server_key_iv: "7fa72116d028967f34320f5287acddbe"
server_key_version: "encryptionkey@0"
I20260808 10:39:17.872083 23656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:17.895859 23656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:17.899219 23656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:17.899449 23656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:17.906832 23656 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.190:40131
I20260808 10:39:17.907131 23729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.190:40131 every 8 connection(s)
I20260808 10:39:17.907845 23656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/data/info.pb
I20260808 10:39:17.909950 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 23656
I20260808 10:39:17.910094 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-0/wal/instance
I20260808 10:39:17.910421 31306 external_mini_cluster.cc:1428] Setting key 0766b047c6e9e99d660052f164281270
I20260808 10:39:17.911057 23730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:17.915342 23730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.146.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.189@KRBTEST.COM" created.
Aug 08 10:39:17 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185557, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:39:17.931238 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:17.918894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.30.146.190:40131)
0808 10:39:17.919258 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:17.919271 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:17.919316 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:39:17.919674 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:17.919677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:17.919874 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:17.920063 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:17.920872 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:17.921811 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:17.922926 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:17.923304 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:17.923428 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:17.926251 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:17.926257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:17.926267 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:17.926286 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:17.927974 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:17.928448 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:17.928461 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:17.928463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:17.928561 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:17.928883 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:17.928886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:17.928888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:17.929362 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:17.929501 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:17.929881 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 10:39:17.930159 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":212,"thread_start_us":105,"threads_started":1}
I20260808 10:39:17.931238 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:17.917851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:40131 (local address 127.0.0.1:58048)
0808 10:39:17.918415 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:17.918457 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:17.919381 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:17.920061 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:17.920067 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:17.920482 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:17.920727 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:17.920740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:17.921946 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:17.921949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:17.922803 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:17.922815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:17.923469 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:17.923476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:17.923637 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:17.924268 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:17.924295 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:17.926086 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:17.928087 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:17.928098 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:17.928107 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:17.928337 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:17.928662 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:17.928665 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:17.928667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:17.928785 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:17.929482 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:17.929488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:17.929624 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 10:39:17.929798 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 10:39:17.930210 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":110,"thread_start_us":177,"threads_started":1}
I20260808 10:39:17.934052 23730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:17.934845 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:17.934524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:46769 (local address 127.0.0.1:47678)
0808 10:39:17.934708 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:17.934780 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:46769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 10:39:17.935117 23674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.146.189:46769: connect: Connection refused (error 111)
I20260808 10:39:17.936123 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:17.935893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:46769 (local address 127.0.0.1:47684)
0808 10:39:17.936001 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:17.936053 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:46769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 10:39:17.936287 23730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.189:46769: Network error: Client connection negotiation failed: client connection to 127.30.146.189:46769: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab.
Entry for principal kudu/127.30.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab.
I20260808 10:39:17.940938 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.189:46769
--webserver_interface=127.30.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab
--principal=kudu/127.30.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:36623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:40131,127.30.146.189:46769,127.30.146.188:40727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:39:17.985562 23730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } attempt: 1
I20260808 10:39:17.986658 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:17.986332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:46769 (local address 127.0.0.1:47690)
0808 10:39:17.986503 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:17.986568 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:46769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 10:39:17.987412 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:17.987184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:46769 (local address 127.0.0.1:47706)
0808 10:39:17.987299 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:17.987353 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:46769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 10:39:17.987535 23730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.189:46769: Network error: Client connection negotiation failed: client connection to 127.30.146.189:46769: connect: Connection refused (error 111)
W20260808 10:39:18.055181 23738 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:39:18.055544 23738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:18.055632 23738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:18.055713 23738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 10:39:18.055812 23730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } attempt: 2
I20260808 10:39:18.056792 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.056389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:46769 (local address 127.0.0.1:47714)
0808 10:39:18.056549 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.056707 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:46769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 10:39:18.057718 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.057443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:46769 (local address 127.0.0.1:47724)
0808 10:39:18.057596 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.057650 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.189:46769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 10:39:18.057871 23730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.189:46769: Network error: Client connection negotiation failed: client connection to 127.30.146.189:46769: connect: Connection refused (error 111)
W20260808 10:39:18.060359 23738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:18.060449 23738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:18.060497 23738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:18.060554 23738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:18.060592 23738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:18.066293 23738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:36623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:40131,127.30.146.189:46769,127.30.146.188:40727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.189.keytab
--principal=kudu/127.30.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.189:46769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:18.067451 23738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:18.068557 23738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:39:18.077488 23744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:18.077466 23746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:18.077466 23743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:18.077800 23738 server_base.cc:1034] running on GCE node
I20260808 10:39:18.078438 23738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:18.079635 23738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:18.080868 23738 hybrid_clock.cc:648] HybridClock initialized: now 1786185558080840 us; error 49 us; skew 500 ppm
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:18.084244 23738 init.cc:377] Logged in from keytab as kudu/127.30.146.189@KRBTEST.COM (short username kudu)
I20260808 10:39:18.085223 23738 webserver.cc:460] Webserver started at http://127.30.146.189:41973/ using document root <none> and password file <none>
I20260808 10:39:18.085803 23738 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:18.085901 23738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:18.086153 23738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:18.088337 23738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/data/instance:
uuid: "9a2e90cecd64478b9092a420f5fce551"
format_stamp: "Formatted at 2026-08-08 10:39:18 on dist-test-slave-9gcw"
server_key: "ab1dd93ea34d05c3a1706d013146d7a4"
server_key_iv: "e89eea626fac69127a8b31161c7a4a3b"
server_key_version: "encryptionkey@0"
I20260808 10:39:18.088910 23738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/wal/instance:
uuid: "9a2e90cecd64478b9092a420f5fce551"
format_stamp: "Formatted at 2026-08-08 10:39:18 on dist-test-slave-9gcw"
server_key: "ab1dd93ea34d05c3a1706d013146d7a4"
server_key_iv: "e89eea626fac69127a8b31161c7a4a3b"
server_key_version: "encryptionkey@0"
I20260808 10:39:18.092763 23738 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:39:18.095221 23753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:18.096436 23738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:39:18.096590 23738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/wal
uuid: "9a2e90cecd64478b9092a420f5fce551"
format_stamp: "Formatted at 2026-08-08 10:39:18 on dist-test-slave-9gcw"
server_key: "ab1dd93ea34d05c3a1706d013146d7a4"
server_key_iv: "e89eea626fac69127a8b31161c7a4a3b"
server_key_version: "encryptionkey@0"
I20260808 10:39:18.096830 23738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:18.123432 23738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:18.127102 23738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:18.127342 23738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:18.134892 23738 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.189:46769
I20260808 10:39:18.135201 23810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.189:46769 every 8 connection(s)
I20260808 10:39:18.136054 23738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/data/info.pb
I20260808 10:39:18.138757 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 23738
I20260808 10:39:18.138929 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-1/wal/instance
I20260808 10:39:18.139061 23811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:18.139315 31306 external_mini_cluster.cc:1428] Setting key 8137f31489672fe98b5a472b1b6cfd8e
I20260808 10:39:18.143579 23811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.146.188@KRBTEST.COM; defaulting to no policy
I20260808 10:39:18.149053 23730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.146.188@KRBTEST.COM" created.
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185557, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:39:18.160806 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.145500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.30.146.190:40131)
0808 10:39:18.145664 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.145669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.146545 (+ 876us) server_negotiation.cc:407] Connection header received
0808 10:39:18.147472 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.147476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.147547 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.147656 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.148964 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.149777 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.151341 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.152341 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.152467 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.156495 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.156499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.156501 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.156526 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.158365 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.158947 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.158951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.158953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.159047 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.159454 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.159458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.159463 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.159746 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.159867 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.160468 (+ 601us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.160609 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:39:18.161124 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.149572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:46769 (local address 127.0.0.1:47738)
0808 10:39:18.149700 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.149715 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:18.149803 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:18.150345 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:18.150348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:18.150542 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:18.150754 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.150762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.151838 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.151844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.152525 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.152540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.153318 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.153322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.153431 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.153858 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:18.153875 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:18.155542 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:18.158517 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.158523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.158529 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.158823 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.159254 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.159256 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.159258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.159347 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.160424 (+ 1077us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:18.160429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:18.160534 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 10:39:18.160622 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:39:18.160777 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 10:39:18.162127 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.145889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:40131 (local address 127.0.0.1:58064)
0808 10:39:18.146370 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.146423 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:18.147261 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:18.147826 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:18.147833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:18.148166 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:18.148800 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.148814 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.149934 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.149937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.151217 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.151226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.152508 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.152513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.152749 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.153685 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:18.153701 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:18.156325 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:18.158509 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.158520 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.158531 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.158805 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.159161 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.159165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.159168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.159332 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.159866 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:18.159872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:18.160157 (+ 285us) client_negotiation.cc:769] Sending connection context
0808 10:39:18.160404 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 10:39:18.160778 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"spinlock_wait_cycles":6912,"thread_start_us":136,"threads_started":1}
I20260808 10:39:18.162132 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.149661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47738 (local address 127.30.146.189:46769)
0808 10:39:18.149964 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.149974 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.150004 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:39:18.150040 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.150044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.150188 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.150315 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.150885 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.151723 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.152655 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.153185 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.153316 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.155816 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.155826 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.155844 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.155884 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.158378 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.158986 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.158995 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.159000 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.159157 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.159925 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.159930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.159935 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.160281 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.160924 (+ 643us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.161043 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.161231 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":130,"threads_started":1}
I20260808 10:39:18.163769 23811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:18.164601 23730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:18.165736 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.165345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33332)
0808 10:39:18.165562 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.165634 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260808 10:39:18.166615 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.166326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33336)
0808 10:39:18.166486 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.166547 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260808 10:39:18.166805 23730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:40727: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab.
Entry for principal kudu/127.30.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab.
I20260808 10:39:18.173579 31306 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
/tmp/dist-test-taskwxv9J_/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.188:40727
--webserver_interface=127.30.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab
--principal=kudu/127.30.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.146.148:36623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.146.190:40131,127.30.146.189:46769,127.30.146.188:40727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:39:18.173658 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.164572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:46769 (local address 127.0.0.1:47748)
0808 10:39:18.164828 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.164845 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:18.164947 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:18.165331 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:18.165334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:18.165490 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:18.165589 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.165594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.166568 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.166574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.167221 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.167230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.167729 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.167733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.168039 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.168550 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:18.168561 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:18.170114 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:18.171958 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.171963 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.171965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.172195 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.172543 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.172548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.172550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.172636 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.173019 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:18.173022 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:18.173109 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 10:39:18.173321 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 10:39:18.173471 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":154}
I20260808 10:39:18.173673 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.164868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.30.146.189:46769)
0808 10:39:18.165020 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.165025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.165049 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:39:18.165084 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.165087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.165197 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.165304 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.165703 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.166465 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.167361 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.167623 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.167733 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.170253 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.170257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.170260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.170287 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.171850 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.172368 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.172372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.172374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.172452 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.172743 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.172747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.172748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.172927 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.173030 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.173378 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.173524 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":77}
I20260808 10:39:18.176553 23811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:18.177536 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.177231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33350)
0808 10:39:18.177344 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.177443 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:39:18.178150 23754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
I20260808 10:39:18.179193 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.178893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33358)
0808 10:39:18.179044 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.179114 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 10:39:18.179381 23811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:40727: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
I20260808 10:39:18.197974 23730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } attempt: 1
I20260808 10:39:18.198925 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.198589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33362)
0808 10:39:18.198775 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.198845 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 10:39:18.199685 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.199480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33370)
0808 10:39:18.199591 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.199636 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 10:39:18.199828 23730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:40727: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
I20260808 10:39:18.228690 23811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } attempt: 1
I20260808 10:39:18.229676 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.229355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33374)
0808 10:39:18.229528 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.229594 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 10:39:18.230502 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.230292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33376)
0808 10:39:18.230392 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.230449 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 10:39:18.230644 23811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:40727: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
I20260808 10:39:18.274998 23730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } attempt: 2
I20260808 10:39:18.275954 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.275648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33386)
0808 10:39:18.275806 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.275870 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 10:39:18.276775 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.276570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33402)
0808 10:39:18.276675 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.276719 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 10:39:18.276924 23730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:40727: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
W20260808 10:39:18.292593 23817 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:39:18.292878 23817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:39:18.292955 23817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:39:18.293030 23817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:39:18.297091 23817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:39:18.297220 23817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:39:18.297292 23817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:39:18.297324 23817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:39:18.297358 23817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:39:18.298811 23811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } attempt: 2
I20260808 10:39:18.299670 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.299375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33406)
0808 10:39:18.299528 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.299591 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 10:39:18.300453 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.300254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33422)
0808 10:39:18.300352 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.300397 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 10:39:18.300595 23811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:40727: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
I20260808 10:39:18.303500 23817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.146.148:36623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.146.190:40131,127.30.146.189:46769,127.30.146.188:40727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/krb5kdc/kudu_127.30.146.188.keytab
--principal=kudu/127.30.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.146.188:40727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:39:18.304639 23817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:39:18.305781 23817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:39:18.314931 23817 server_base.cc:1034] running on GCE node
W20260808 10:39:18.314993 23827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:18.314842 23824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:39:18.314898 23823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:39:18.315825 23817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:39:18.316959 23817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:39:18.318163 23817 hybrid_clock.cc:648] HybridClock initialized: now 1786185558318145 us; error 61 us; skew 500 ppm
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:39:18.321336 23817 init.cc:377] Logged in from keytab as kudu/127.30.146.188@KRBTEST.COM (short username kudu)
I20260808 10:39:18.322410 23817 webserver.cc:460] Webserver started at http://127.30.146.188:37785/ using document root <none> and password file <none>
I20260808 10:39:18.323005 23817 fs_manager.cc:359] Metadata directory not provided
I20260808 10:39:18.323108 23817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:39:18.323351 23817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:39:18.325553 23817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/data/instance:
uuid: "f2d0eacbd0d44d0a9a4e593c65585d14"
format_stamp: "Formatted at 2026-08-08 10:39:18 on dist-test-slave-9gcw"
server_key: "80add1bcd6385953f24bd57a4475ea96"
server_key_iv: "6a1f8d8efda4e8c21427949ac04742a2"
server_key_version: "encryptionkey@0"
I20260808 10:39:18.326157 23817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/wal/instance:
uuid: "f2d0eacbd0d44d0a9a4e593c65585d14"
format_stamp: "Formatted at 2026-08-08 10:39:18 on dist-test-slave-9gcw"
server_key: "80add1bcd6385953f24bd57a4475ea96"
server_key_iv: "6a1f8d8efda4e8c21427949ac04742a2"
server_key_version: "encryptionkey@0"
I20260808 10:39:18.330135 23817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:39:18.332919 23834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:39:18.334394 23817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:39:18.334555 23817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/data,/tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/wal
uuid: "f2d0eacbd0d44d0a9a4e593c65585d14"
format_stamp: "Formatted at 2026-08-08 10:39:18 on dist-test-slave-9gcw"
server_key: "80add1bcd6385953f24bd57a4475ea96"
server_key_iv: "6a1f8d8efda4e8c21427949ac04742a2"
server_key_version: "encryptionkey@0"
I20260808 10:39:18.334702 23817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:39:18.370098 23817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:39:18.373588 23817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:39:18.373805 23817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:39:18.376083 23730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } attempt: 3
I20260808 10:39:18.376916 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.376590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33426)
0808 10:39:18.376708 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.376798 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 10:39:18.377749 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.377536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33432)
0808 10:39:18.377630 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.377684 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 10:39:18.377957 23730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.146.188:40727: Network error: Client connection negotiation failed: client connection to 127.30.146.188:40727: connect: Connection refused (error 111)
I20260808 10:39:18.381284 23817 rpc_server.cc:307] RPC server started. Bound to: 127.30.146.188:40727
I20260808 10:39:18.381558 23888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.146.188:40727 every 8 connection(s)
I20260808 10:39:18.382323 23817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/data/info.pb
I20260808 10:39:18.385076 23889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:39:18.388840 23889 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:18.391783 23811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } attempt: 3
I20260808 10:39:18.392987 31306 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu as pid 23817
I20260808 10:39:18.393133 31306 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwxv9J_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185256684052-31306-0/minicluster-data/master-2/wal/instance
I20260808 10:39:18.393532 31306 external_mini_cluster.cc:1428] Setting key aa87fb96fc127379d861ff506e5fc0bc
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.189@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
I20260808 10:39:18.406025 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.390813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58066 (local address 127.30.146.190:40131)
0808 10:39:18.390968 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.390973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.391795 (+ 822us) server_negotiation.cc:407] Connection header received
0808 10:39:18.393266 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.393270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.393337 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.393460 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.394949 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.395729 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.397360 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.397749 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.397885 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.401080 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.401087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.401091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.401166 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.402993 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.403843 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.403850 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.403856 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.403975 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.404457 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.404463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.404465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.404767 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.404882 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.405666 (+ 784us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.405815 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:39:18.406118 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.392350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33438)
0808 10:39:18.392605 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.392623 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:18.392721 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:18.393550 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:18.393554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:18.393776 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:18.393910 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.393919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.396077 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.396081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.396604 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.396613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.397482 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.397487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.397621 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.398263 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:18.398283 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:18.400087 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:18.404198 (+ 4111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.404205 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.404210 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.404482 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.404835 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.404841 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.404844 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.404950 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.405567 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:18.405573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:18.405680 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 10:39:18.405790 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:39:18.405940 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260808 10:39:18.406905 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.391084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.190:40131 (local address 127.0.0.1:58066)
0808 10:39:18.391589 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.391661 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:18.392984 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:18.393518 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:18.393529 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:18.394059 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:18.394768 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.394789 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.395886 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.395890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.397185 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.397205 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.398056 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.398059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.398245 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.398837 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:18.398869 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:18.400894 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:18.403174 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.403184 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.403196 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.403676 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.404141 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.404148 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.404154 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.404335 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.404872 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:18.404879 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:18.405298 (+ 419us) client_negotiation.cc:769] Sending connection context
0808 10:39:18.405566 (+ 268us) client_negotiation.cc:240] Negotiation successful
0808 10:39:18.405921 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":6,"thread_start_us":115,"threads_started":1}
I20260808 10:39:18.406906 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.392457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.30.146.188:40727)
0808 10:39:18.392806 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.392820 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.392865 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:39:18.393021 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.393042 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.393328 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.394054 (+ 726us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.394563 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.395962 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.396726 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.397291 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.397492 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.400271 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.400279 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.400296 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.400328 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.404070 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.404597 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.404605 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.404607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.404702 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.405063 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.405066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.405067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.405437 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.406403 (+ 966us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.406511 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.406751 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":235,"spinlock_wait_cycles":40192,"thread_start_us":153,"threads_started":1}
I20260808 10:39:18.406906 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.394202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33448 (local address 127.30.146.188:40727)
0808 10:39:18.394506 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.394516 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.394804 (+ 288us) server_negotiation.cc:407] Connection header received
0808 10:39:18.394999 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.395004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.395054 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.395183 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.396186 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.397063 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.398213 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.398444 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.398571 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.402016 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.402021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.402026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.402053 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.404368 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.404871 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.404876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.404880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.404957 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.405339 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.405342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.405344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.405537 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.405715 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.406558 (+ 843us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.406672 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":241,"thread_start_us":124,"threads_started":1}
I20260808 10:39:18.408208 23889 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:18.411258 23811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551: Bootstrap starting.
I20260808 10:39:18.413781 23811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551: Neither blocks nor log segments found. Creating new log.
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
I20260808 10:39:18.414578 23811 log.cc:824] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:18.416981 23811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551: No bootstrap required, opened a new log
I20260808 10:39:18.418782 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.408708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.189:46769 (local address 127.0.0.1:47752)
0808 10:39:18.409009 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.409027 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:18.409182 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:18.409663 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:18.409667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:18.409871 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:18.409991 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.409999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.410883 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.410891 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.411613 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.411624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.412163 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.412168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.412296 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.412872 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:18.412891 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:18.414488 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:18.416819 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.416823 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.416826 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.417112 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.417559 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.417562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.417563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.417663 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.418176 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:18.418180 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:18.418290 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 10:39:18.418414 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 10:39:18.418579 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234}
I20260808 10:39:18.418792 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.408863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47752 (local address 127.30.146.189:46769)
0808 10:39:18.409013 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.409022 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.409093 (+ 71us) server_negotiation.cc:407] Connection header received
0808 10:39:18.409346 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.409349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.409402 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.409463 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.410176 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.410753 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.411755 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.412032 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.412169 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.414669 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.414674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.414679 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.414736 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.416482 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.417295 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.417299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.417301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.417400 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.417791 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.417795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.417797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.418037 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.418182 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.418495 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.418629 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:39:18.419752 23889 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:39:18.421559 23811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } }
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.190@KRBTEST.COM
I20260808 10:39:18.421803 23811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a2e90cecd64478b9092a420f5fce551, State: Initialized, Role: FOLLOWER
I20260808 10:39:18.422147 23811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } }
I20260808 10:39:18.422714 23898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } } }
I20260808 10:39:18.423256 23898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:18.423604 23811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185558, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.188@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
I20260808 10:39:18.428067 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.415134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.30.146.190:40131)
0808 10:39:18.415276 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.415279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.415290 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:39:18.415339 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.415341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.415377 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.415473 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.416545 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.417406 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.418460 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.418800 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.418876 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.422490 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.422495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.422500 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.422536 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.424660 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.425312 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.425316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.425318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.425464 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.425895 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.425900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.425901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.426178 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.426447 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.427699 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.427847 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 10:39:18.430593 23911 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:18.430678 23911 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:18.430696 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.420415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33456)
0808 10:39:18.420517 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.420535 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:18.420619 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:18.420929 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:18.420933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:18.421195 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:18.421319 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.421332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.422451 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.422457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.423174 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.423185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.423944 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.423947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.424023 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.424518 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:18.424538 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:18.426078 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:18.428583 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.428587 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.428589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.428879 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.429283 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.429287 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.429289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.429402 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.430104 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:18.430107 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:18.430208 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 10:39:18.430331 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 10:39:18.430475 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 10:39:18.431172 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.420584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.30.146.188:40727)
0808 10:39:18.420719 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.420724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.420737 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:39:18.420773 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.420782 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.420818 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.420916 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.421472 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.422323 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.423359 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.423718 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.423813 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.426228 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.426232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.426235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.426272 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.428469 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.429007 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.429010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.429012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.429103 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.429537 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.429541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.429543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.429930 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.430060 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.430959 (+ 899us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.431062 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:39:18.427827 23738 master_runner.cc:186] Error getting master registration for 127.30.146.190:40131: OK, instance_id { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" instance_seqno: 1786185557904706 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:39:18.431679 23738 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:39:18.434140 23889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14: Bootstrap starting.
I20260808 10:39:18.436877 23889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14: Neither blocks nor log segments found. Creating new log.
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.146.189@KRBTEST.COM
I20260808 10:39:18.437758 23889 log.cc:824] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:18.441054 23889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14: No bootstrap required, opened a new log
I20260808 10:39:18.441875 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.430599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47758 (local address 127.30.146.189:46769)
0808 10:39:18.430715 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.430719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.430731 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:39:18.430776 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.430779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.430824 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.430895 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.432188 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.432928 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.433999 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.434316 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.434419 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.437687 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.437691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.437693 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.437721 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.439638 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.440190 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.440194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.440196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.440289 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.440656 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.440659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.440661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.440903 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.441110 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.441426 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.441547 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:39:18.445861 31306 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:39:18.445957 23889 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } }
I20260808 10:39:18.446231 23889 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d0eacbd0d44d0a9a4e593c65585d14, State: Initialized, Role: FOLLOWER
I20260808 10:39:18.446617 23889 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } }
I20260808 10:39:18.447290 23914 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } } }
I20260808 10:39:18.447435 23914 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:18.448215 23889 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:18.447293 23699 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "04f3b7229a524598b81f313eb2bd5ec0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58082
W20260808 10:39:18.454535 23925 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:18.454617 23925 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:18.452889 23817 master_runner.cc:186] Error getting master registration for 127.30.146.190:40131: OK, instance_id { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" instance_seqno: 1786185557904706 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:39:18.456955 23817 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:39:18.542147 23730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } attempt: 4
Aug 08 10:39:18 dist-test-slave-9gcw krb5kdc[23643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185557, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.146.190@KRBTEST.COM for kudu/127.30.146.188@KRBTEST.COM
I20260808 10:39:18.556514 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.542852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.146.188:40727 (local address 127.0.0.1:33468)
0808 10:39:18.543082 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 10:39:18.543102 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:39:18.543273 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:39:18.543715 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:39:18.543719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:39:18.543932 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:39:18.544038 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.544045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.545285 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.545290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.545882 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:39:18.545892 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.546563 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.546575 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:39:18.546672 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.547345 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:39:18.547372 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:39:18.549460 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:39:18.551917 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.551921 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.551924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.553666 (+ 1742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.554603 (+ 937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:39:18.554608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:39:18.554610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:39:18.554725 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:39:18.555838 (+ 1113us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:39:18.555842 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:39:18.555976 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 10:39:18.556123 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 10:39:18.556296 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260808 10:39:18.556514 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.542928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33468 (local address 127.30.146.188:40727)
0808 10:39:18.543092 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.543100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.543196 (+ 96us) server_negotiation.cc:407] Connection header received
0808 10:39:18.543407 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.543411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.543476 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.543566 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.544189 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.545131 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.546032 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.546438 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.546559 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.549638 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.549642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.549645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.549685 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.551756 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.554333 (+ 2577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.554338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.554340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.554455 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.554871 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.554874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.554876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.555728 (+ 852us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.555851 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.556189 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.556328 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:39:18.560135 23730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0: Bootstrap starting.
I20260808 10:39:18.562795 23730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0: Neither blocks nor log segments found. Creating new log.
I20260808 10:39:18.563943 23730 log.cc:824] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:39:18.566479 23730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0: No bootstrap required, opened a new log
I20260808 10:39:18.570133 23730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } }
I20260808 10:39:18.570415 23730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f3b7229a524598b81f313eb2bd5ec0, State: Initialized, Role: FOLLOWER
I20260808 10:39:18.570809 23730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } }
I20260808 10:39:18.571530 23928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } } }
I20260808 10:39:18.571880 23928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:18.572419 23730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:39:18.572474 23699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 10:39:18.572584 23699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:18.574210 23699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } }
I20260808 10:39:18.575423 23699 leader_election.cc:288] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [CANDIDATE]: Term 1 election: Requested vote from peers 9a2e90cecd64478b9092a420f5fce551 (127.30.146.189:46769), f2d0eacbd0d44d0a9a4e593c65585d14 (127.30.146.188:40727)
I20260808 10:39:18.576604 23862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04f3b7229a524598b81f313eb2bd5ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14"
I20260808 10:39:18.576608 23783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04f3b7229a524598b81f313eb2bd5ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9a2e90cecd64478b9092a420f5fce551"
I20260808 10:39:18.576897 23783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:18.576896 23862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:39:18.578504 23783 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f3b7229a524598b81f313eb2bd5ec0 in term 1.
I20260808 10:39:18.578505 23862 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f3b7229a524598b81f313eb2bd5ec0 in term 1.
I20260808 10:39:18.579221 23674 leader_election.cc:302] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f3b7229a524598b81f313eb2bd5ec0, 9a2e90cecd64478b9092a420f5fce551; no voters:
I20260808 10:39:18.579604 23928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:39:18.579954 23928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [term 1 LEADER]: Becoming Leader. State: Replica: 04f3b7229a524598b81f313eb2bd5ec0, State: Running, Role: LEADER
I20260808 10:39:18.580444 23928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } }
I20260808 10:39:18.582679 23928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f3b7229a524598b81f313eb2bd5ec0. Latest consensus state: current_term: 1 leader_uuid: "04f3b7229a524598b81f313eb2bd5ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } } }
I20260808 10:39:18.582877 23928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [sys.catalog]: This master's current role is: LEADER
W20260808 10:39:18.587774 23940 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:39:18.587870 23940 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:39:18.588130 23656 master_runner.cc:263] No leader master found from master 04f3b7229a524598b81f313eb2bd5ec0
I20260808 10:39:18.588200 23656 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:39:18.588328 23947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:39:18.589582 23947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 10:39:18.625283 23840 tablet.cc:2369] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:39:18.629061 23759 tablet.cc:2369] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:39:18.633831 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.589094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.30.146.189:46769)
0808 10:39:18.589250 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.589256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.597541 (+ 8285us) server_negotiation.cc:407] Connection header received
0808 10:39:18.598460 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.598465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.598533 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.598602 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.605349 (+ 6747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.606174 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.611245 (+ 5071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.611657 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.611756 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.620931 (+ 9175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.620937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.620940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.620980 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.623052 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.629377 (+ 6325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.629382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.629384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.629501 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.630649 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.630654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.630656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.630943 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.631096 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.633449 (+ 2353us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.633615 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:39:18.635471 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.589624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58086 (local address 127.30.146.190:40131)
0808 10:39:18.589756 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.589762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.597801 (+ 8039us) server_negotiation.cc:407] Connection header received
0808 10:39:18.599929 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.599933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.600021 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.600100 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.603129 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.603961 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.609847 (+ 5886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.610247 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.610343 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.623715 (+ 13372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.623720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.623725 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.623764 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.625886 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.630158 (+ 4272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.630163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.630165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.630274 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.631979 (+ 1705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.631983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.631985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.632267 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.632444 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.635104 (+ 2660us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.635263 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:39:18.636726 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:18.588491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33484 (local address 127.30.146.188:40727)
0808 10:39:18.588645 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.588651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.598355 (+ 9704us) server_negotiation.cc:407] Connection header received
0808 10:39:18.600970 (+ 2615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.600975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.601041 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.601126 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.607631 (+ 6505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.608487 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.612526 (+ 4039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:18.612918 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:18.613012 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:18.626752 (+ 13740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:39:18.626757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:39:18.626760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:39:18.626799 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:39:18.628791 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.631481 (+ 2690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.631485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.631487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.631596 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:39:18.632820 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:39:18.632825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:39:18.632827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:39:18.633106 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:39:18.633295 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:18.636374 (+ 3079us) server_negotiation.cc:299] Negotiation successful
0808 10:39:18.636538 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:39:18.639762 23862 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [term 1 FOLLOWER]: Refusing update from remote peer 04f3b7229a524598b81f313eb2bd5ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:18.640712 23941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:18.645882 23914 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f3b7229a524598b81f313eb2bd5ec0. Latest consensus state: current_term: 1 leader_uuid: "04f3b7229a524598b81f313eb2bd5ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } } }
I20260808 10:39:18.646037 23914 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:18.647192 23783 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [term 1 FOLLOWER]: Refusing update from remote peer 04f3b7229a524598b81f313eb2bd5ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:39:18.647863 23928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:39:18.651041 23928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04f3b7229a524598b81f313eb2bd5ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } } }
I20260808 10:39:18.651201 23928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:18.652086 23898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f3b7229a524598b81f313eb2bd5ec0. Latest consensus state: current_term: 1 leader_uuid: "04f3b7229a524598b81f313eb2bd5ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } } }
I20260808 10:39:18.652206 23898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:18.652418 23953 mvcc.cc:205] Tried to move back new op lower bound from 7316216048087523328 to 7316216047950344192. Current Snapshot: MvccSnapshot[applied={T|T < 7316216048087523328}]
I20260808 10:39:18.652571 23914 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04f3b7229a524598b81f313eb2bd5ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } } }
I20260808 10:39:18.652720 23914 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:18.654510 23928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04f3b7229a524598b81f313eb2bd5ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } } }
I20260808 10:39:18.654600 23952 mvcc.cc:205] Tried to move back new op lower bound from 7316216048087523328 to 7316216047950344192. Current Snapshot: MvccSnapshot[applied={T|T < 7316216048087523328}]
I20260808 10:39:18.654616 23928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:39:18.655577 23898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04f3b7229a524598b81f313eb2bd5ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f3b7229a524598b81f313eb2bd5ec0" member_type: VOTER last_known_addr { host: "127.30.146.190" port: 40131 } } peers { permanent_uuid: "9a2e90cecd64478b9092a420f5fce551" member_type: VOTER last_known_addr { host: "127.30.146.189" port: 46769 } } peers { permanent_uuid: "f2d0eacbd0d44d0a9a4e593c65585d14" member_type: VOTER last_known_addr { host: "127.30.146.188" port: 40727 } } }
I20260808 10:39:18.655683 23898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:39:18.656922 23947 catalog_manager.cc:1347] Generated new cluster ID: dbfd6492fb054802915eafaed3de3d21
I20260808 10:39:18.657002 23947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:39:18.658469 23955 mvcc.cc:205] Tried to move back new op lower bound from 7316216048087523328 to 7316216047950344192. Current Snapshot: MvccSnapshot[applied={T|T < 7316216048087523328}]
I20260808 10:39:18.682466 23947 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:39:18.683622 23947 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:39:18.690799 23947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04f3b7229a524598b81f313eb2bd5ec0: Generated new TSK 0
I20260808 10:39:18.691658 23947 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:39:18.721104 23893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.719441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.30.146.188:40727)
0808 10:39:18.719570 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.719575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.719704 (+ 129us) server_negotiation.cc:407] Connection header received
0808 10:39:18.719934 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.719940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.720001 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.720057 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.720959 (+ 902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33498: BlockingRecv error: recv got EOF from 127.0.0.1:33498 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 10:39:18.721104 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.719567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.30.146.189:46769)
0808 10:39:18.719714 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.719721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.719834 (+ 113us) server_negotiation.cc:407] Connection header received
0808 10:39:18.720005 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.720008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.720062 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.720112 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.720959 (+ 847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47776: BlockingRecv error: recv got EOF from 127.0.0.1:47776 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 10:39:18.721210 23735 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.719013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58092 (local address 127.30.146.190:40131)
0808 10:39:18.719165 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.719170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.720007 (+ 837us) server_negotiation.cc:407] Connection header received
0808 10:39:18.720265 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.720270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.720330 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.720395 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.721016 (+ 621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58092: BlockingRecv error: recv got EOF from 127.0.0.1:58092 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 10:39:18.726517 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.723649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47788 (local address 127.30.146.189:46769)
0808 10:39:18.723779 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.723782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.725515 (+ 1733us) server_negotiation.cc:407] Connection header received
0808 10:39:18.725694 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.725696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.725773 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.725844 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.726380 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47788: BlockingRecv error: recv got EOF from 127.0.0.1:47788 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 10:39:18.730690 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.729813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47794 (local address 127.30.146.189:46769)
0808 10:39:18.729932 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.729935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.729947 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:39:18.729987 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.729989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.730028 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.730093 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.730578 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47794: BlockingRecv error: recv got EOF from 127.0.0.1:47794 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:39:18.736004 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.734891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47802 (local address 127.30.146.189:46769)
0808 10:39:18.735001 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.735011 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.735159 (+ 148us) server_negotiation.cc:407] Connection header received
0808 10:39:18.735296 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.735298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.735347 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.735412 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.735901 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47802: BlockingRecv error: recv got EOF from 127.0.0.1:47802 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 10:39:18.743221 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.742034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.30.146.189:46769)
0808 10:39:18.742215 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.742220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.742360 (+ 140us) server_negotiation.cc:407] Connection header received
0808 10:39:18.742478 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.742481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.742543 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.742643 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.743103 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47804: BlockingRecv error: recv got EOF from 127.0.0.1:47804 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
W20260808 10:39:18.754060 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.752892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.30.146.189:46769)
0808 10:39:18.753047 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.753053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.753176 (+ 123us) server_negotiation.cc:407] Connection header received
0808 10:39:18.753343 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.753348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.753419 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.753540 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.753934 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47808: BlockingRecv error: recv got EOF from 127.0.0.1:47808 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 10:39:18.773396 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.772089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47820 (local address 127.30.146.189:46769)
0808 10:39:18.772272 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.772278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.772412 (+ 134us) server_negotiation.cc:407] Connection header received
0808 10:39:18.772607 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.772612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.772679 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.772778 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.773269 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47820: BlockingRecv error: recv got EOF from 127.0.0.1:47820 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 10:39:18.808887 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.807597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.30.146.189:46769)
0808 10:39:18.807805 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.807814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.807872 (+ 58us) server_negotiation.cc:407] Connection header received
0808 10:39:18.808040 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.808046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.808146 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.808274 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.808744 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47826: BlockingRecv error: recv got EOF from 127.0.0.1:47826 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 10:39:18.876605 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:18.875088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.30.146.189:46769)
0808 10:39:18.875272 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:18.875278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:18.875475 (+ 197us) server_negotiation.cc:407] Connection header received
0808 10:39:18.875662 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:18.875666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:18.875742 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:18.875837 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:18.876446 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47842: BlockingRecv error: recv got EOF from 127.0.0.1:47842 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 10:39:19.008742 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:19.007247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47846 (local address 127.30.146.189:46769)
0808 10:39:19.007468 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:19.007472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:19.007712 (+ 240us) server_negotiation.cc:407] Connection header received
0808 10:39:19.007861 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:19.007863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:19.007915 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:19.008007 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:19.008618 (+ 611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47846: BlockingRecv error: recv got EOF from 127.0.0.1:47846 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260808 10:39:19.268249 23815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:39:19.267032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47854 (local address 127.30.146.189:46769)
0808 10:39:19.267179 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:19.267182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:19.267273 (+ 91us) server_negotiation.cc:407] Connection header received
0808 10:39:19.267390 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:19.267393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:19.267444 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:19.267533 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:39:19.268143 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47854: BlockingRecv error: recv got EOF from 127.0.0.1:47854 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:39:19.432153 23911 catalog_manager.cc:1259] Loaded cluster ID: dbfd6492fb054802915eafaed3de3d21
I20260808 10:39:19.432260 23911 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551: loading cluster ID for follower catalog manager: success
I20260808 10:39:19.434216 23911 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551: acquiring CA information for follower catalog manager: success
I20260808 10:39:19.435357 23911 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9a2e90cecd64478b9092a420f5fce551: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:39:19.455889 23925 catalog_manager.cc:1259] Loaded cluster ID: dbfd6492fb054802915eafaed3de3d21
I20260808 10:39:19.455963 23925 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14: loading cluster ID for follower catalog manager: success
I20260808 10:39:19.457628 23925 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14: acquiring CA information for follower catalog manager: success
I20260808 10:39:19.458770 23925 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2d0eacbd0d44d0a9a4e593c65585d14: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:39:19.787062 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:19.782786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47866 (local address 127.30.146.189:46769)
0808 10:39:19.783042 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:19.783046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:19.783062 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:39:19.783147 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:19.783149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:19.783278 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:19.783379 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:19.783798 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:19.784331 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:19.785444 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:19.785668 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:19.785748 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:19.786087 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:19.786548 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:19.786643 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:19.786798 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:39:19.786852 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":96,"threads_started":1}
I20260808 10:39:19.794224 24040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:19.790371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33514 (local address 127.30.146.188:40727)
0808 10:39:19.790592 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:19.790595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:19.790609 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:39:19.790742 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:19.790745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:19.790871 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:19.790968 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:19.791244 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:19.791745 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:19.792743 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:19.792965 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:19.793036 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:19.793440 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:19.793838 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:19.793930 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:19.794057 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 10:39:19.794103 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":80,"threads_started":1}
I20260808 10:39:19.802666 24040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:19.798175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.30.146.188:40727)
0808 10:39:19.798337 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:19.798342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:19.798623 (+ 281us) server_negotiation.cc:407] Connection header received
0808 10:39:19.798767 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:19.798770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:19.798819 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:19.798896 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:19.799328 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:19.799830 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:19.801195 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:19.801584 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:19.801729 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:19.802096 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:19.802198 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:19.802277 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:19.802452 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:39:19.802518 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 10:39:19.802827 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:19.797933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47874 (local address 127.30.146.189:46769)
0808 10:39:19.798103 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:19.798108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:19.798629 (+ 521us) server_negotiation.cc:407] Connection header received
0808 10:39:19.798900 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:19.798904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:19.798970 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:19.799072 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:19.799244 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:19.800171 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:19.801465 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:19.801812 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:19.801936 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:19.802263 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:19.802354 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:19.802449 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:19.802584 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 10:39:19.802653 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:39:19.803478 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:39:19.798184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58094 (local address 127.30.146.190:40131)
0808 10:39:19.798447 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 10:39:19.798454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:39:19.798479 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:39:19.798563 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:39:19.798565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:39:19.798759 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:39:19.798912 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:39:19.799225 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:19.800150 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:19.801521 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:39:19.801891 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:39:19.802276 (+ 385us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:39:19.802440 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:39:19.802834 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:39:19.803233 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0808 10:39:19.803273 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 10:39:19.803336 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":87,"threads_started":1}
I20260808 10:39:19.811887 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 23656
I20260808 10:39:19.821220 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 23738
I20260808 10:39:19.829677 31306 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwxv9J_/build/debug/bin/kudu with pid 23817
2026-08-08T10:39:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3348 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6601 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (303134 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST